Some argue that writing multiple choice questions do not effectively evaluate one’s comprehension of the material provided in a training course. Multiple choice questions, if written with intelligence and thought, can be used effectively to evaluate one’s knowledge. Writing a quality assessment takes time, research, and needs to be managed like any project.
In many organizations, a subject matter expert (SME) may be tasked with writing the assessment. Later the SME hands off those terribly written questions to instructional designer. As the instructional designer, don’t just grumble about it, instead, reach out to assist your SME and educate them on how to write better questions so that next time the questions reach your desk, you will have an easier time deciphering the question and distractors.
Before the SME begins writing the assessment, consult with them on how to write effective questions and you will be surprised of the quality of the questions you receive. Explain to them how the question, answer, and distracters must be relevant to the content and make sense. This is where you as the instructional designer can make a difference in the quality and effectiveness of an assessment.
Here is a list of guidelines I use to help my SMEs write better questions:
- Write the questions after the training content is complete, correct, edited and approved.
- Write questions that directly challenge the content. In other words, don’t write questions for content not covered in the training material.
- No True/False or Yes/No type questions. Instead, write a question that provides some thought such as “From the following statements, which two provide positive results?” Or “From the following items, which one provides a return on investment?”
- Avoid “all of the above” or “none of the above” answers. Instead, use wording such as “From the following, which two statements are correct”.
- Never write a question that requires a student to remember the wrong answer. For example, “From the following statements, which two are incorrect?” You are asking the student to recall what is wrong rather than challenge them on a correct method or task.
- For each multiple choice question, provide a total of four answers per question with the following credentials: provide one or two correct answers, provide two or three credible distractor answers (no give-a-way answers). This is where many quizzes or tests fail miserably. Providing nonsense answers does not challenge a student’s knowledge and shows laziness in writing a qualified question. Perhaps it is an indication of a bad question. If you can’t provide credible distractor answers, try writing a new question.
- Finally, make sure the answers are 100% correct or 100% incorrect. Don’t write questions that are “more” or “less” correct than others—they must be either right or wrong.
Help your subject matter experts in these areas, and you will see great dividends when it comes time for you to develop the final assessment. As training professionals, promote your knowledge of assessment development within your organization. Let others know that you are an expert and provide real value to your company and training reputation.
For examples of how I approach questions written by SMEs. See Also Writing Assessment Questions #1 or search for “Writing Assessment Questions” on this site.