These pieces originally appeared as a weekly column entitled “Lessons” in The New York Times between 1999 and 2003.
[ THIS ARTICLE FIRST APPEARED IN THE NEW YORK TIMES ON JULY 18, 2001 ]
In standardized tests, standards vary
In Ohio this spring, 98 percent of students who took the state’s graduation exam passed. In California, less than half passed a similar test.
Surely, young people aren’t that much smarter in Ohio than in California. It’s just that passing is a murky concept.
Many classroom quizzes define passing as good answers on 60 percent of the questions. But there are tough and easy tests. Fifty percent on a hard exam can mean more than 70 percent on an easy one.
The education bills now before Congress will require all public-school students to pass state tests. That means little because states can set passing scores at whatever level they like.
California decided that correct answers to 55 percent of its questions would pass in math but 60 percent in English. Newspapers complained that despite such low standards, most students still failed – even with a “D” acceptable.
But the public has no way to judge if a test is so difficult that 55 percent should be enough to pass.
After a test is given, states like Ohio and California convene panels to review the questions, with the easier ones — those that more students got right — listed first. Panelists, mostly teachers, then negotiate about where to place a “bookmark” separating the minimum number of questions a student should answer properly.
California’s experience this year is instructive. Its 100 panelists, chosen from volunteers, got little guidance. They were given a dictionary definition of “competence” (ready for “necessities and conveniences of life”) and told to take the long view – don’t worry about setting a high passing score because some day more students will meet it.
Panelists privately assigned passing points, then met in small groups to seek agreement. Opinions varied substantially.
The panel’s collective recommendation was that passing should require about 70 percent of questions answered properly. But California’s schools superintendent, Delaine Eastin, overruled this. The state adopted her proposal of 55 percent for math and 60 percent for English.
The decision was pure politics; Ms. Eastin thought too many students would fail under the panel’s rule, and the reaction would then sink her testing program. As a gauge of how students should perform, 55 percent or 60 percent should be taken no more seriously than 70 percent.
Similar approaches are used elsewhere. Both the House and Senate versions of the education bill, now in conference committee, require every child to read by third grade. Each state will decide what this means. If panelists place their bookmarks after questions of only rudimentary skill, states will claim all third graders can read. If panelists establish higher cut-off points, states will report failure. Yet pupils in failing states may read better than those in successful ones.
Leaving this to Washington may be no better. The National Assessment Governing Board, which is appointed by Congress and the president, uses panels of educators, business leaders, politicians and other citizens to define basic, proficient or advanced scores on a federal test, the National Assessment of Educational Progress.
Many experts believe these definitions have been set unreasonably high to spur change by exaggerating the failures of schools.
Mark Musick, chairman of the governing board, does not concede that the national levels are wrong, but agrees that there is no consensus about where they should be set. Mr. Musick has urged state officials, without success, to meet with each other and the national board to seek equivalent passing points on the many tests now being used.
Anomalies abound. In Massachusetts, only 28 percent of eighth graders were proficient on a state science exam, but the national test showed their performance was as good as or better than students in every nation except Singapore.
Mr. Musick noted that in North Carolina only 20 percent of eighth graders were proficient on the national math exam, but the state test found 68 percent proficient. Other states have similar discrepancies. Differences in subjective judgments of panelists are the main cause.
We frequently complain that more than two-thirds of fourth graders “can’t read,” meaning they are below what the governing board panels say is proficient on the national test. But on an international survey of reading ability, American fourth graders scored higher than pupils everywhere except Finland.
The national assessment also shows that only 2 percent of seniors are advanced in math. But nearly twice that number pass advanced placement exams in math. More than 10 percent of all seniors score above 600 on the College Board’s SAT (or achieve an equivalent score on the ACT), a level probably reflecting advanced skill.
Of course, someone must decide when students are proficient. But judgments are inevitably personal and, in today’s climate, subject to political whim. Passing or failing a standardized test is less straightforward than it seems.
