If you are reading this, I can safely assume that you have taken a multiple-choice test at some point
in your life. You’ve most
likely
taken more than you can count.
Unfortunately, there is a very strong
probability that you’ve encountered numerous test questions (or "items,"
as they may be called) that were so
poorly designed you didn’t need to possess any related knowledge to guess
the correct answer, or you were able to at least narrow the correct
answer to two choices. Chances are also good that you’ve
read poorly worded
test items that created confusion for you as the
test-taker.
Even with those limitations, test-givers highly
prefer multiple-choice tests due to the ease and efficiency of
administration. The
familiarity of the test structure also makes these tests a comfortable
experience for test-takers.
While multiple-choice tests provide a traditional
format for test-takers and a quick record of training for test-givers,
they are often ineffective in measuring knowledge due to poor test item
writing. Most of us think
that writing a test question is easy because we’ve read thousands (maybe
tens of thousands) of test questions and responses during our lifetimes. But contrary to what many people think, multiple-choice items are
not easy to write if your tests are to be valid (testing the thing you
want to test) and reliable (offering dependable results every time). If you are an employer who relies on multiple-choice testing to
assess knowledge, properly constructed test items are essential.
Multiple-choice test questions include stems and responses:
- The stem is the question asked or incomplete statement provided.
- The key is the correct response.
-
The distractor is the incorrect response.
(Each test item usually has multiple distractors but could have only one.)
Good test item writing provides:
- Stems that ask only one direct question
- Stems that provide enough wording to be clear but not so much that unnecessary information is included
- Stems that grammatically match all of their responses
- Responses that grammatically follow their stems
- Responses that are similar in length and structure
- Stems that include most of the wording with responses that are limited in length
- Stems that avoid absolutes, vague references, or negatives that can confuse the test-taker (Examples of what to avoid include words like "none," "all," "usually." Also, avoid using phrases with negatives like, “Which of the following does not include…”)
Good test items also provide:
- Distractors that are plausible to anyone with limited knowledge although obviously incorrect to someone with the appropriate knowledge (For example, don’t offer implausible answers like “Mickey Mouse” to questions about people. Every distractor should sound logical to a layperson.)
-
Distractors that incorporate common
procedural mistakes (For
example, if employees frequently use the wrong form to complete a task
that is being tested,
be sure to include the erroneously selected form as a legitimate
distractor in your responses.)
- Distractors that avoid absolutes that can trick the test-taker or provide a clue (Using terms like "always" and "never" can be confusing or can provide clues to the answer.)
- Distractors that avoid vague options (Answers like “typically” and “possibly” are confusing and imprecise, and therefore are unfair to the test-taker.)
- An appropriate number of distractors (Nowhere is it written that multiple-choice items must provide 4 responses. Actually, in most cases three responses [one key plus two distractors] provide the best test item validity*.)
How do your test items stack up to these
item-writing tips? Properly
worded test items help to ensure validity and reliability from your
tests, so it’s worth the effort to examine your testing tools and refine
them as needed.
In my next blog we will examine some sample test
questions and how we might improve them.
*Rodriguez, Michael C./ Three options are optimal for multiple-choice items : A meta-analysis of 80 years of research. In: Educational Measurement: Issues and Practice. 2005 ; Vol. 24, No. 2. pp. 3-13.