Beware High-Stakes Testing

Before I begin, a few disclaimers. I'm not opposed to tests, not even multiple choice tests like the SAT which, for many high schoolers, are extremely high-stakes. National, well-researched tests are important, and the results can be meaningful and ought to inform practice at every level. I'm not opposed to judging schools and teachers--in part--on students' test scores. But test data, like anything else, must be looked at critically before running with it. And that has not happened many times.

Today, the New York Times reported on what should really be called a scandal for the New York Department of Education. Test scores were artificially inflated by making the tests easier to pass. When critics of the Bloomberg/Klein regime pointed this out over a year ago, the Times and the state looked the other way. Repeated claims of rising test scores should have been accompanied by the note that the benchmarks are moving, but instead, credit-claiming was all that we heard. It was obvious that test scores were rising due, at least in part, to manipulations of the test and the scoring, but the mainstream media remained silent.

Every year, test scores in many states rise and rise, but that only reflects manipulation of the tests! A few intelligent folks have pointed this out in the past: Diane Ravitch comes to mind. Here she notes that No Child Left Behind actually hurt student achievement and slowed student progress, particularly for racial minorities.

I'm pleased that the state of New York, at least, has finally realized the disservice they are doing when they advertise that an outrageously high percentage of students are "proficient" when, in truth, many high school students still are unable to read or do basic math. The "Shock Plunge" in test scores is actually not a plunge at all; instead, it's a realization that calling a kid proficient doesn't actually raise numeracy or literacy.

In defense of the state board of regents, they have been warning us for months that this was coming. The raising of proficiency levels for state tests has indeed been a long time coming, but the warning disguises the fact that the proficiency level for years was decreased specifically so that folks like Joel Klein could brag about improving test scores year after year, despite the fact that actual student achievement had remained constant or even decreased. It's dishonest, embarrassing and shameful, and those responsible ought to be held accountable.

from the NYT story:

And the results could cast doubts on the city’s improvements over the past several years; both the mayor and the schools chancellor, Joel I. Klein, have used increases in state test scores as evidence that schools have improved.

“It certainly complicates the Bloomberg administration message, because the state test is completely unreliable,” said Michael J. Petrilli, a researcher with the Fordham Institute, a Washington-based research group.

Just unconscionable. This is why tests should be just one measure of success for students, teachers, and schools. Test scores--clearly so malleable to the point of being almost meaningless--should only be considered as one part of a larger review. If tests are too important, they are likely to be manipulated, and that's exactly what we've seen in New York. The one major takeaway for ed reformers from this story: pay attention! Don't believe everything you here, and hold on to a healthy dose of skepticism.