These pieces originally appeared as a weekly column entitled “Lessons” in The New York Times between 1999 and 2003.
[THIS ARTICLE FIRST APPEARED IN THE NEW YORK TIMES ON MAY 1, 2002]
States Teeter When Balancing Standards With Tests
Critics of standardized tests lament that teachers are now being compelled to “teach to the test.” Most people are befuddled by this complaint. As President Bush said, if we are teaching math and reading, teaching to math and reading tests makes sense.
But the critics do have something of a point. Math and reading include many distinct skills. If a test gauges only a few of the skills regarded as important, teachers may emphasize only those that are tested, ignoring other skills that are just as crucial.
Almost every state has adopted higher standards and tests to see if students meet them. Most officials, however, don’t know if their tests are any good. So some states have asked Achieve Inc., a nonprofit consulting firm, to evaluate some of their tests. Achieve’s board includes governors and executives, led by John Engler, governor of Michigan, and Louis V. Gerstner Jr., chairman of I.B.M.
Achieve has found that most tests are poorly matched to state standards. As a result, states are requiring one set of skills while creating incentives to teach a different set. Of all the tests it studied, Achieve found that only the Massachusetts 10th grade tests were relatively well-aligned with the state’s standards. (Achieve’s evaluations can be seen at www.achieve.org.)
There are two criteria for judging if tests and standards are aligned, but states typically fulfill only one. First, every test question must assess a skill actually found in the standards. This mostly happens.
Second, every required standard must be assessed, either by tests, student work samples or other evaluations. Over all, skills should have the same relative importance in tests as in standards. Otherwise, teachers will have incentives to give less emphasis to skills not found on tests.
States have mostly failed to keep this balance between standards and test questions. Consider a Minnesota fifth-grade reading standard that expects pupils to identify both the main idea and supporting details in a passage. That is a good standard; if Minnesota assessed it, teaching to its test would indeed teach reading.
But the state’s test emphasizes identification of details, not the main idea. A test with this flaw might have a passage about Harriet Tubman but then have pupils identify her home state (Maryland) without asking them to articulate the theme of the passage: how the underground railroad advanced anti-slavery struggles.
Educators call identifying details a “basic skill.” Discerning a main idea is a “higher-order skill.” Teachers preparing for tests mostly of basic skills will not do much to train students in higher-order thinking.
Jennifer Vranek, now director of a group that promotes higher standards in Washington State, oversaw most of Achieve’s studies. Ms. Vranek said that state tests typically overemphasized basic skills. Every state has a middle school geometry standard about measuring figures like triangles, squares, prisms and cones. But tests ask mainly about the simpler forms, like triangles and squares.
Most people think that if test scores are low, basic skills should be a priority. After all, it seems, students should master the basics first. But good teaching lets basic and higher skills reinforce each other. With the Tubman passage, a child need not perfect an ability to recall details before learning to summarize a main idea.
Or consider a basic skill like two-digit multiplication. Students who can multiply may still make errors and fail test questions. But students can move on to somewhat more advanced skills, like solving simple algebra equations, while continuing to practice multiplying. If tests focus mostly on basics, teachers may assign too much multiplication and too little algebra. Higher scores will make it seem, wrongly, that students met state standards.
Misalignment mostly results from wanting tougher standards on the cheap. It costs more to test complex skills than basic ones. Many standards cannot be assessed with inexpensive, mostly multiple-choice tests, even if the tests also ask for a few short-essay answers.
Wisconsin’s literacy standard, for example, expects high school students to prepare a research paper that detects bias in sources. Checking if students meet this standard requires specialists to grade work samples, and it requires trusting teacher evaluations of class projects. Instead, Wisconsin tests literacy mostly by having students infer what reading passages mean. This skill is important, but only part of the standard.
Achieve’s analysts have concluded that standards and tests have gotten better, and that more improvement lies ahead. Perhaps so, but until states create tests that truly test the standards, teachers who complain about having to teach to the test should be given a fuller hearing.
