Developing Multiple Choice Tests

If you are reading this, I can safely assume that you have taken a multiple-choice test at some point in your life. You’ve most likely taken more than you can count. 

Unfortunately, there is a very strong probability that you’ve encountered numerous test questions (or "items," as they may be called) that were so poorly designed you didn’t need to possess any related knowledge to guess the correct answer, or you were able to at least narrow the correct answer to two choices. Chances are also good that you’ve read poorly worded test items that created confusion for you as the test-taker. 

Even with those limitations, test-givers highly prefer multiple-choice tests due to the ease and efficiency of administration. The familiarity of the test structure also makes these tests a comfortable experience for test-takers. 

While multiple-choice tests provide a traditional format for test-takers and a quick record of training for test-givers, they are often ineffective in measuring knowledge due to poor test item writing. Most of us think that writing a test question is easy because we’ve read thousands (maybe tens of thousands) of test questions and responses during our lifetimes. But contrary to what many people think, multiple-choice items are not easy to write if your tests are to be valid (testing the thing you want to test) and reliable (offering dependable results every time). If you are an employer who relies on multiple-choice testing to assess knowledge, properly constructed test items are essential.

Multiple-choice test questions include stems and responses: 

-          The stem is the question asked or incomplete statement provided.

-          The key is the correct response.

-          The distractor is the incorrect response. (Each test item usually has multiple distractors but could have only one.)

Good test item writing provides:

-          Stems that ask only one direct question

-          Stems that provide enough wording to be clear but not so much that unnecessary information is included

-          Stems that grammatically match all of their responses

-          Responses that grammatically follow their stems

-          Responses that are similar in length and structure

-          Stems that include most of the wording with responses that are limited in length

-          Stems that avoid absolutes, vague references, or negatives that can confuse the test-taker (Examples of what to avoid include words like "none," "all," "usually." Also, avoid using phrases with negatives like, “Which of the following does not include…”)

Good test items also provide:

-          Distractors that are plausible to anyone with limited knowledge although obviously incorrect to someone with the appropriate knowledge (For example, don’t offer implausible answers like “Mickey Mouse” to questions about people.  Every distractor should sound logical to a layperson.)

-          Distractors that incorporate common procedural mistakes (For example, if employees frequently use the wrong form to complete a task that is being tested, be sure to include the erroneously selected form as a legitimate distractor in your responses.)

-          Distractors that avoid absolutes that can trick the test-taker or provide a clue (Using terms like "always" and "never" can be confusing or can provide clues to the answer.)

-          Distractors that avoid vague options (Answers like “typically” and “possibly” are confusing and imprecise, and therefore are unfair to the test-taker.)

-          An appropriate number of distractors (Nowhere is it written that multiple-choice items must provide 4 responses. Actually, in most cases three responses [one key plus two distractors] provide the best test item validity*.)

How do your test items stack up to these item-writing tips? Properly worded test items help to ensure validity and reliability from your tests, so it’s worth the effort to examine your testing tools and refine them as needed. 

In my next blog we will examine some sample test questions and how we might improve them.

*Rodriguez, Michael C./ Three options are optimal for multiple-choice items : A meta-analysis of 80 years of research. In: Educational Measurement: Issues and Practice. 2005 ; Vol. 24, No. 2. pp. 3-13.

Return to the home page.

Return to the blog list.