 |
| An SAT Question: Step by Step |
 |
 |
|
|
|
It takes about two years for a question to go from conception to final publication on the test. Here's an inside look at the vetting process |
|
|
|
|
|
|
|
|
|
|
 |
|
|
|
Posted Sunday, October 19, 2003
Step 1: Writing It
An ETS staff member in the math department writes a question and submits it for review.
Step 2: Reviewing it
The question is read by other ETS staff members in the math department, who make sure there is only one possible answer and that the problem can be solved by a student who takes a typical high school math curriculum. Another group of staffers check that there's an even balance between male and female references and that no ethnic group is disparaged. Finally, they make sure there is nothing upsetting in the question—no references to war or violence or to something like suicide or cancer that might distract a student recently touched by such an event. This one passes
all those hurdles.
Step 3: Trying it Out
Pretesting is done by
putting the question on a nonscored section of the SAT. When the test results come in, ETS staff members check the question's performance in several areas. One thing they monitor is the difficulty level. Some questions are intended to be easier to answer than others; the drafters estimate that a large proportion of students will answer this one correctly, and indeed 85% of students do,
confirming its low degree of difficulty. It is put in the pool of available questions.
Step 4: Placing it on the test
ETS staffers select 60 questions from the preapproved pool for the math test. They want a good mix of degrees of difficulty and skills being tested. The position of the correct answers also needs to be varied so that not too many questions have a correct answer of E, as this
one does.
Step 5: Vetting the test
Members of the ETS math staff, an outside panel of high school and college teachers, and a wider group of College Board members review the test to see if the questions can be clearly understood. In this case, minor language revisions are suggested.
Step 6: Making Final Changes
The ETS math-test developers go through the questions and comments from the review committees, making any adjustments they deem necessary. This question requires a change to match an earlier revision in its accompanying chart.
Step 7: The Big Test
This question, seen below, actually appeared on the May 2000 version of the math test. After the test came back, the question's scores were analyzed to make sure its performance matched how it did on the pretest. It performed pretty much the same way—84% answered it correctly. Because it has now been disseminated publicly, it will never appear on the domestic test again.
|
