These pieces originally appeared as a weekly column entitled "Lessons" in The New York Times between 1999 and 2003.

[ THIS ARTICLE FIRST APPEARED IN THE NEW YORK TIMES ON JUNE 6, 2001 ]

Test here and there, not everywhere

By Richard Rothstein

The federal government spends nearly $20 billion a year to aid schools and should ensure that the money is well spent. President Bush wants to do so by requiring states to test third- to eighth- grade students each year and intervene if scores lag.

States may use any test to track progress, the president says. Some may buy commercial exams; others may design their own. But to prevent cheating (with easier exams each year), Mr. Bush also wants states to confirm their gains by giving a federal fourth- and eighth-grade exam, the National Assessment of Educational Progress, or N.A.E.P.

N.A.E.P. (pronounced nape) has reported state-by-state scores since 1990, and Mr. Bush reasons that if a state test is valid, whatever progress it reflects should also appear on the national test.

The Senate bill agrees; the House one does not. Instead, it lets states give any two tests, saying higher scores on both would verify success.

The Senate is half-right and the House has it backward. With the federal exam now voluntary for states, Mr. Bush's demand that each state use it is smart. But beyond this, federally mandated tests are not needed. To ensure that its dollars are well spent, the government can get data it needs from N.A.E.P. alone.

Only a random sample of pupils takes the federal test, and each participant takes only some questions, so there are no individual scores.

Surprisingly, this is a more accurate statewide gauge than full tests for every child. Statisticians can calculate results, like how urban Hispanic pupils who get certain questions right are likely to do on questions they were not given. The distribution of scores and averages for such students can be estimated, as if all took the entire exam.

Such sampling avoids "teaching to the test" because the range of essential topics is so wide. For example, fourth-grade math covers 56 areas, from using decimal points to calculating volume. It would take days to ask enough questions on each to know if students mastered the full curriculum.

Standardized exams solve this by quizzing all pupils on only some topics. It is costly to change tests, so teachers soon learn what will be tested. They can then review topics that raise scores, ignoring other areas.

The National Assessment has another solution: giving a short test in which each pupil has different questions. Since only about 2,500 students may be scientifically sampled in a state, no teacher expects her students to be selected and she cannot know which topics will be on any exam. She must teach a balanced curriculum so performance will be acceptable no matter what is tested.

N.A.E.P. sampling also permits better questions than typical state tests that need short and cheap-to- grade items.

A state test might ask fourth graders to select an answer to "six times three" from several choices, easily scanned for scoring. But more complex tests would ask for illustration of the problem. A good reply might be that if six houses each had three  doors, there would be 18 doors in all. A common incorrect response might be that one house had six doors and another had three. Trained readers must be paid to score such answers.

State tests include fewer items like this than the federal exam because one test covering a full curriculum cannot include many such questions in a few hours. The National Assessment touches on all topics by giving some pupils more difficult questions while asking others simply to recognize that 6 x 3 = 18. Statisticians can project how each student might handle the items not given.

N.A.E.P. can be improved. It tells how black students perform, but race predicts achievement less well than parental education. A state's black students may score lower if fewer parents finished high school and are less able to help their children. More should be done to perfect how the exam identifies subgroups, but commercial tests have similar flaws.

Some House members object that the National Assessment implies a national curriculum that the government should avoid. This argument is wrong because the test samples such broad subjects that states are not pressured to focus on any topic.  States with different emphases can excel, provided their approaches are reasonable.

If each state gives the National Assessment and reports subgroup scores, we will know which improve. With other tests so inferior to N.A.E.P., nothing is gained by forcing states to use them. Of course, states wanting to test annually for school or student-level accountability can do so.

But some states say fewer tests are better. Whichever system a state chooses, if the nation sees scores improve, no further restriction of local control is needed. And the National Assessment is the best way — really the only way — of knowing if gains truly occur.

Return to the Education Column Archive