Auto-graded Test Question Writing

Alignment

Quality Matters

4.3 C The course content is appropriate to the reading level of the intended learners.

4.6 C The course content strives to reflect a culturally diverse perspective that is free from bias.

8.3 C The course provides accessible text and images in files, documents, LMS pages, and web pages to meet the needs of diverse learners.

National Standards for Quality Online Teaching

G1 The online teacher chooses appropriate assessment tools, which allow students the opportunity to demonstrate mastery of the content.

Staying Aligned with Objectives

Questions must assess the stated learning objective for the assessment and have sufficient lesson content for students to be able to choose the correct answer.

A common mistake when scripting assessments is to stray from the stated learning objectives. To avoid this mistake, you can restate the learning objectives in the test directions- the same objectives should also guide each scripted test question.

Example:

In this lesson, you learned about flow states: optimal states of consciousness where we feel and perform our best. In this quiz, you will describe flow states, and give an example of when one has occurred in your own life.
Note: The learning objective has been bolded and italicized for this example.

Audience

Test questions should be written following the guidelines for the intended audience. 

• Style Guide: SLS Audience

Avoid “trick” responses, unnecessary humor, analogies, or bias.

Quiz Pools

• Script multiple-choice questions with four choices
  • 1 correct answer, 3 distractors
    • Note: Accommodated versions of each quiz will be made where one distractor will be removed. If there is a reason your course may need fewer than 3 distractors, please consult with your project lead.
      • Example: WL course may use MC format for true/false in target language (cierto/falso)
• Test questions will be pulled from a pool so each question must contain all the information needed to answer and be independent of each other
  • Note: If several questions refer to one artifact (ex. analyze a story passage), the artifact would be included in the part directions, not each individual question.
• Style Guide: Quiz Pools

• University of Southern Australia: Writing Effective Quiz Questions (see Table on Page 2 for Blooms with question types)
• Michigan Assessment Consortium Common Assessment: Module 8 (Writing Selected-Response Items)
  • Wayne RESA Item Review: Multiple Choice (from the above module)
• Wiley: Writing Quiz Questions
• Style Guide: Alignment
• Style Guide: Quality Matters Rubric
• Style Guide: SLS Audience
• Style Guide: Quiz Pools
• Style Guide: Quality Matters Recertification
• Style Guide: Objectives
• Style Guide: Quiz / Test Directions