Write your reflections by selecting an idea from the reading, describing your thoughts and feelings about it (Total of 1 FUL

IT’S TIME TO DIG INTO TEST-CONSTRUCTION AND THE INNARDS OF THE ITEMS THAT

actually make up educational tests. In this brief chapter, and the two

chapters following it, I’ll be dispensing a series of rules to follow if

you wish to create a set of spectacular items for your own classroom

tests. Don’t forget that there are some solid references cited at the end

of each chapter. If you want to dig deeper into the rules for test-

development and item-construction, I encourage you to explore one

or more of the listed resources. Prior to picking up your shovel for any

diligent item-digging, though, I need to remind you of some prelim-

inary considerations you’ll need to attend to before you’ve con-

structed even a single item.

All About Inferences Since the early pages of this book, you’ve been reading that teachers

use tests in order to make inferences about their students’ cognitive

or affective status. And, once those score-based inferences have been

made, the teacher then reaches instructional decisions based (at least

in part) on those inferences. Educational assessment revolves around

inference making.

5 An Introduction to Test Building

6 0

ch5.qxd 7/10/2003 9:57 AM Page 60

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 6 1

Well, if that is true (and you’ve already read it so often in this

book, you must know that it’s true), then why not focus on your

intended score-based inferences throughout all of your test-

development efforts? If you do so, your tests are more likely to gen-

erate data that will support valid inferences, and you are more likely

to make better instructional decisions.

But what does this inference-focused approach to test-

construction look like in the real world? Well, it means that before

you begin to even think about a test and what it might look like, you

first isolate the instructional decisions that need to be made. What are

you going to use this test for? Will you be looking at the results to see

if students have mastered a given curricular aim? Will you be looking

at the results to find out what your students’ current geometric

understandings are so that you can decide on the most suitable con-

tent for your upcoming geometry unit? Perhaps, as is often the case,

you’ll be testing your students merely to assign grades for a given unit

of study or, perhaps, for an entire term.

Clearly, the purposes for classroom tests can vary substantially,

and the decisions linked to those purposes will also vary. But all of

these purposes will be better satisfied if subsequent decisions are

based on more accurate score-based inferences about students’ cogni-

tive or affective status. This three-step process is represented graphi-

cally in Figure 5.1, where you can see that prior to test-construction, a

teacher should strive to identify the decisions that will be riding on

the yet-to-be-built test’s results and what sorts of score-based infer-

ences will best contribute to the teacher’s pre-identified decisions.

If you keep these two considerations—decisions and

contributing-inferences—constantly in mind, often these considera-

tions will shape the nature of the test itself. For instance, if you’re try-

ing to decide what to address instructionally, based on students’ entry

capabilities, you’ll need substantially fewer test items than you would

need if you were trying to decide what students’ end-of-course grades

should be. If your goal is to make decisions regarding your own

ch5.qxd 7/10/2003 9:57 AM Page 61

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R6 2

instructional effectiveness, you’ll need more items still—enough to

support reasonable inferences about what parts of your instructional

program seemed to work and what parts didn’t. The decision-at-issue,

and the inferences that can best inform that decision, should always

govern the particulars of a teacher’s classroom assessments.

Don’t forget that the most important kind of validity for class-

room tests is content-related evidence of validity. Prior to test-

building, a wonderful opportunity arises to make sure that the cur-

ricular aims to be assessed (the skills and knowledge to be taught) are

satisfactorily represented in a classroom test. There should be no

obvious content gaps, and the number and weighting of items on a

test should be representative of the importance of the content stan-

dards being measured.

Almost all of these test-design decisions that a teacher makes are

judgment calls. There are no sacrosanct 10 commandments of test

construction, no inviolate rules about the numbers of needed items

or the necessary breadth of content coverage. However, teachers who

give serious attention to the preliminary questions reflected in Figure

5.1, namely, (1) the instructional decisions-at-issue and (2) the score-

based inferences that best support those inferences, will be more likely

to create a decent classroom test that helps improve their instruction.

5 . 1 THREE STEPS IN CLASSROOM TEST BUILDING

ch5.qxd 7/10/2003 9:57 AM Page 62

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 6 3

Surely such teachers’ tests will be meaningfully better than the class-

room tests devised by teachers who immediately dive into item-

development without first giving some thought to the upcoming

decisions or inferences they’ll need to make.

Two Item-Types There are all sorts of tidy little distinctions in the field of educational

measurement. We’ve talked about the differences between an apti-

tude test and an achievement test. We have considered three varieties

of validity evidence as well as three types of reliability. Assessment

folks, it seems, love to conjure up categories into which measurement

commodities can be tossed. The following two chapters are, in fact,

based on another of those category schemes: a two-way division

between the kinds of items that make up all educational tests.

Every item that nestles in an educational test can be classified

either as a selected-response item or as a constructed-response item. The

labels for these two item-categories do a pretty decent job of describ-

ing the key feature of the items contained in each category. A selected-

response item calls for students to select an answer from a set of pre-

sented options. Multiple-choice items are usually the first sort of

selected-response item that comes to most folks’ minds. But True-

False items are also members of the selected-response family because

the test-taker must choose, that is, select, between two presented

options, namely, True or False.

In contrast, a constructed-response item requires test-takers to

respond by constructing, that is, generating, an answer, an essay, or

whatever the item calls for. The most common kinds of constructed-

response items are essay items and short-answer items. In each of

those instances, the student must create a response, not merely

choose a response from a set of already provided alternatives.

There are few teachers who haven’t heard the usual critiques

of selected-response and constructed-response items. It is often said

that selected-response items, while easy to score, usually deal with

ch5.qxd 7/10/2003 9:57 AM Page 63

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R6 4

low-level, readily memorizable content. Just as often it is said that

constructed-response items, while eliciting higher-level cognitive re-

sponses from students, take far more time to score and are difficult to

score with adequate objectivity. Like most things in life, there are

clear trade-offs involved in the construction of classroom tests when

choosing between selected-response and constructed-response items.

There is simply no body of assembled research indicating that

one of these item-types is superior to the other. Clearly, it all depends.

It depends on the curricular aims being measured. It depends on how

much energy and time a teacher has available to score students’

responses to constructed-response items. It depends on how skillful

the teacher is in devising different sorts of test items. Generally speak-

ing, teachers must first look carefully at the curricular aims to

be sought, then try to develop tests—either selected-response or

constructed-response—that seem most likely to yield valid score-

based inferences about their students.

Irrespective of whether you end up favoring selected-response

items, constructed-response items, or a hybrid mix of each, there are

some experience-based guidelines that will help you dodge the most

vile of the test-development shortcomings classroom teachers are apt

to encounter.

Roadblocks to Good Item-Writing In Chapter 6, I’ll be considering item-construction rules for selected-

response tests. In Chapter 7, I’ll deal with rules for constructing and

scoring students’ performances on constructed-response tests. But

first, I want share five obstacles to good item-writing that apply to

both selected-response items and constructed-response items. These

five roadblocks interfere with the purpose of test-items: to permit ac-

curate inferences about students’ status. They are (1) unclear directions,

(2) ambiguous statements, (3) unintentional clues, (4) complex phrasing,

and (5) difficult vocabulary. Let’s run through each of these five obsta-

cles. I’ll supply an example of each of these roadblocks. Your job as a

ch5.qxd 7/10/2003 9:57 AM Page 64

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 6 5

test-developer is to dodge all five obstacles. Such dodging can often

be difficult.

Unclear Directions. You’d be surprised how often students fail to

perform well on a classroom test simply because they don’t really

know what they’re supposed to do. The sorts of shoddy direction-

giving that students often encounter in the tests they take are illus-

trated in the following directions for a classroom midterm exam.

These directions may be fictitious, but they are strikingly similar to

the kinds of directions most students see week in and week out.

DREADFUL DIRECTIONS

Directions: This midterm exam consists of four parts. Each part

contains different types of items, for example, short-answer and

multiple-choice items. Each of the exam’s four parts covers one

of the major units you have studied thus far during the course.

Work your way through the items efficiently because there is a

time limit for exam-completion. Do your best. Good luck!

These fictitious directions are, indeed, dreadful because they give stu-

dents no guidance about the weighting of the exam’s four parts.

Because some of the four parts contain both selected-response items

and constructed-response items, it certainly is possible that the

teacher might regard particular parts of the exam as more important

than other parts of the exam. Nonetheless, these directions don’t give

students a clue about differential weighting.

Notice, too, that the directions allude to a need for efficiency

because of the italicized “time limit.” What is that time limit? Should

students devote equal time to the four parts which, based on these

dismal directions, might contain equal or different numbers of items?

Given these illustrative directions, students simply don’t know the

answers to a number of important issues that they will surely face

during the exam.

ch5.qxd 7/10/2003 9:57 AM Page 65

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R6 6

As the creator of your own classroom tests, you will typically have

a very thorough understanding of how you’re expecting your stu-

dents to respond. Remember, though, that most of your students

cannot read your mind. (Fear those who can!) To make sure the direc-

tions to your tests are clear and complete, try to put yourself “inside

the head” of one of your students, perhaps a less-swift student, and

then silently read your test’s directions from that student’s perspec-

tive. If what a test-taker is supposed to do appears to be even slightly

opaque, then spruce up your directions until their meaning is

unmistakable.

Ambiguous Statements. Unless you are a diplomat, ambiguity is

something to be avoided. It’s especially reprehensible in educational

tests, where it can be found in hazy directions (as in the preceding

item-writing roadblock) and, even more critically, in the items them-

selves. Again, ambiguous statements usually appear because teachers

“know what they mean” when they write an item. Students, unfor-

tunately, are not in on that secret.

Consider the following ambiguous True-False item:

AN AMBIGUITY-LADEN TRUE-FALSE ITEM

T F Several research studies show that adults often

become domineering toward young children

because of their inherited characteristics.

Does the “their” in this item refer to characteristics inherited by the

domineering adults or to the characteristics inherited by the young

children?

This is actually an easy item to fix. All the teacher needs to do is

replace the ambiguity-inducing pronoun with the name of the group

to which it refers. Faulty-reference pronouns are a common cause of

ambiguity, as are words or phrases that might have double meanings

in the context of the item.

ch5.qxd 7/10/2003 9:57 AM Page 66

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 6 7

Unintentional Clues. Sometimes teachers accidentally include clues

that help less knowledgeable students appear more knowledgeable. A

frequent example in multiple-choice tests is when teachers’ consis-

tently make the correct answer-option longer than the other answer-

options. Typically, the extra length results from teachers’ incorporat-

ing qualifiers to ensure that the correct answer is absolutely correct.

Whatever the reason, if you routinely make your correct answers

longer than your incorrect answers, all but your truly clucko students

will figure out what’s going on.

Another fairly common flaw in teacher-written multiple-choice

items occurs when there is a grammatical tip-off regarding which

answer-option is the winning answer-option. Consider the following

example dealing with biological terminology and you’ll see what I

mean.

A GRAMMATICALLY CLUED GIVE-AWAY ITEM

The commonly recognized example of a pachyderm is an

a. elephant.

b. turtle.

c. lion.

d. pigeon.

Yes, the use of the article “an” in the first part of the item makes it

clear that the first letter of the correct answer needs to be a vowel.

And because “elephant” is the only answer-choice that fills the bill, it

is embarrassingly obvious that choice a is the correct answer. An easy

way to fix such an item would have been to put all the needed

articles in the answer choices so that those choices would be

(a) an elephant, (b) a turtle, (c) a lion, and (d) a pigeon.

Another common instance of unintentional clue dispensing

occurs when teachers toss a “never” or “always” into the false state-

ments of True-False items. Most students will recognize that there are

ch5.qxd 7/10/2003 9:57 AM Page 67

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R6 8

few absolutes in life, so it’s prudent to choose a “false” response for

any items containing “never,” “always,” or “absolutely.”

Inadvertent clues muck up the accuracy of score-based inferences

by making it appear that some students have mastered a given cur-

ricular aim when, in reality, they haven’t. The more inadvertent clues

that you allow to creep into your classroom tests, the more “false pos-

itives” you will have on your hands. As before, prior to administering

a test, review all items carefully to see if there are any aspects of an

item that tip off the correct response.

Complex Phrasing. There are ways to say things simply, and there

are ways to say things opaquely. In item-writing, simplicity wins and

opacity loses. Beware of lengthy sentences in your items—sentences

that begin to resemble a novella. Also, if your items contain so many

whos or whiches that most sentences require a pronoun-eradicator,

then simplify. We don’t want students coming up with incorrect

responses because they couldn’t untangle an item’s complicated

construction.

See the snarled syntax in the following illustrative history item:

A CONSUMMATELY COMPLEX RIGHT-WRONG ITEM

Right or Wrong: Having been established following World War II

in a patent ploy to accomplish that which the League of

Nations failed to carry out subsequent to World War I, namely,

peace-preservation, the United Nations (headquartered in

New York) has, on a number of occasions, taken part in armed

peacekeeping interventions throughout various parts of the

world.

There is probably a core idea nestled somewhere in this comma-laden

catastrophe of a sentence, but it is truly tough to figure out what that

core idea is. How can students determine if such a meandering

statement is right or wrong when they can’t detect what the

ch5.qxd 7/10/2003 9:57 AM Page 68

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 6 9

statement is really saying? When you churn out your test items, strive

for simplicity.

Difficult Vocabulary. Test items are not the place for teachers to

display their verbal sophistication or boost their self-esteem through

the use of high-toned words. Skilled writers will always pitch their

vocabulary at a level apt to be understood by their readers.

Remember, the readers of your tests will be students—students whose

vocabularies probably don’t match your own. Therefore, eschew

polysyllabic verbiage in your items. In fact, eschew phrases such as

“eschew polysyllabic verbiage.”

Here’s an example of a high-school language arts item (10th grade

literature) with a vocabulary level that’s altogether too high:

A POLYSYLLABICALLY INUNDATED MULTIPLE-CHOICE ITEM

Considering the quintessential phenomenological attribute

evinced by Mrs. Watkins in the short story you just read, which

of the following options best characterizes that attribute?

a. superabundant garrulousness

b. a paucity of profundity

c. hyperbolic affectations

d. mellifluous articulation

If this teacher wants students to be able to discern what makes the fic-

tional Mrs. Watkins tick, why not ask them in language that normal

folks can comprehend?

Wrap Up Looking back at this brief introduction to building your own class-

room tests, you’ll hopefully recall that the central mission of all such

assessment is (1) to help you make valid inferences about your stu-

dents so you can then (2) make better decisions about how to instruct

those students. Never, never create a test just to be creating a test.

ch5.qxd 7/10/2003 9:57 AM Page 69

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

T E S T B E T T E R , T E A C H B E T T E R7 0

Always focus on the instructional decisions that are at issue and,

based on those decisions, try to isolate the sorts of inferences about

students you’ll need to arrive at in order to make those decisions

more defensibly. Beyond the mission of making inferences about

your students based on their test performances, there is no other rea-

son to test those students.

I have also indicated that when it comes time to put together

your own classroom tests, the items you’ll be using are always going

to be, by definition, either selected-response items, constructed-

response items, or a combination of those two item-types. There are

advantages and disadvantages of both item-types.

Finally, I offered one hopefully useful test-construction sugges-

tion. For every test that you create, review all items and directions

from the perspective of your students. When you reconsider your

items “using students’ eyes,” you’ll almost always be able to improve

your tests. And by “improve” them, I mean make them better instru-

ments for uncovering students’ covert cognitive and affective status.

INSTRUCTIONALLY FOCUSED TESTING TIPS

• Do not commence any test-construction activity until you have

isolated, with genuine clarity, (1) the instructional decisions-at-

issue and (2) the score-based inferences that will best support

those decisions.

• Judiciously employ selected-response items and constructed-

response items so that your test-based inferences are likely to be

valid.

• Avoid the five roadblocks to good item-writing: unclear direc-

tions, ambiguous statements, unintentional clues, complex

phrasing, and difficult vocabulary.

ch5.qxd 7/10/2003 9:57 AM Page 70

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

A n I n t r o d u c t i o n t o T e s t B u i l d i n g 7 1

Recommended Resources

Linn, R. L., & Gronlund, N. E. (2000). Measurement and assessment in teaching (8th ed.). Upper Saddle River, NJ: Merrill.

McMillan, J. H. (2001). Classroom assessment: Principles and practice for effective instruction (2nd ed.). Boston: Allyn & Bacon.

Northwest Regional Educational Laboratory. (1991). Paper-and-pencil test devel- opment [Videotape]. Los Angeles: IOX Assessment Associates.

Stiggins, R. J. (Program Consultant). (1996). Assessing reasoning in the class- room: A professional development video [Videotape]. Portland, OR: Assessment Training Institute.

Stiggins, R. J. (2001). Student-involved classroom assessment (4th ed.). Upper Saddle River, NJ: Prentice Hall.

ch5.qxd 7/10/2003 9:57 AM Page 71

Popham, W. J. (2003). Test better, teach better : The instructional role of assessment. Association for Supervision & Curriculum Development. Created from amridge on 2022-01-20 14:31:44.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n &

C u rr

ic u lu

m D

e ve

lo p m

e n t. A

ll ri g h ts

r e se

rv e d .

In this chapter, I’ll be describing selected-response items, focusing on

the three main types: (1) binary-choice items, (2) matching items, and

(3) multiple-choice items. I’ll first take a quick peek at the advantages

and disadvantages of each item-type and then list a set of rules for

constructing that kind of item.

Binary-Choice Items A binary-choice item is one in which the test-taker must choose

between only two options. The most common kind of binary-choice

item is the True-False item, in which students are given a set of state-

ments and then asked to indicate whether each statement is true or

false.

Perhaps you’ve never encountered the phrase “binary-choice”

before, and you suspect I’m trying to impress you by tossing out an

esoteric assessment term. What I want you to realize is that you can

build all sorts of decent test items by presenting students with a two-

choice challenge. The student’s choices, for instance, could be

between right and wrong, correct and incorrect, accurate or inaccurate,

and so on. If you think of binary-choice items only as True-False, you

may overlook some useful dichotomous-item possibilities.

6 Selected-Response Items

7 2

ch6.qxd 7/10/2003 9:57 AM Page 72

Popham, W. James. Test Better, Teach Better : The Instructional Role of Assessment, Association for Supervision & Curriculum Development, 2003. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/amridge/detail.action?docID=5704436. Created from amridge on 2022-01-20 14:33:07.

C o p yr

ig h t ©

2 0 0 3 . A

ss o ci

a tio

n f o r

S u p e rv

is io

n & </p

Collepals.com Plagiarism Free Papers

Are you looking for custom essay writing service or even dissertation writing services? Just request for our write my paper service, and we'll match you with the best essay writer in your subject! With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Get ZERO PLAGIARISM, HUMAN WRITTEN ESSAYS

Why Hire Collepals.com writers to do your paper?

Quality- We are experienced and have access to ample research materials.

We write plagiarism Free Content

Confidential- We never share or sell your personal information to third parties.

Support-Chat with us today! We are always waiting to answer all your questions.