Thanks to the federal No Child Left Behind act and other high-stakes testing laws, schoolteachers and sumo wrestlers have something in common. According to Steven Levitt and Stephen J. Dubner, authors of the 2005 best-selling Freakonomics: A Rogue Economist Explores the Hidden Side of Everything, they both cheat.
This article is limited to the freaky economics of high-stakes testing in public schools. If you’re interested in learning more about why sumo wrestlers cheat, why drug dealers still live with their mothers, how the legalization of abortion lowered crime rates, or how much parents really matter, a little instant cash or a small personal loan is all that stands between you and your very own paperback copy of Freakonomics. It’s easy for almost anyone — even someone with bad credit — interested in reading about global cooling, patriotic prostitutes, and why suicide bombers should buy life insurance to fund the purchase of Levitt and Dubner’s 2009 hardcover book SuperFreakonomics with an installment loan.
Laudable if laughable goals
The federal government mandated high-stakes testing as part of the No Child Left Behind law signed by President Bush in 2002. The theory behind high-stakes testing is that setting high standards and establishing measurable goals improves individual outcomes in education. The NCLB law requires states to develop tests of basic skills to be administered to all students in certain grades as a prerequisite for federal funding.
Strong incentives to cheat
For as long as there have been tests, Levitt and Dubner point out, students have been motivated to cheat. But high-stakes testing has significantly increased the incentives for cheating on the part of teachers. In a high-stakes testing regime, if students test well, teachers may be praised or promoted. In some states, they may even receive bonuses. At one time, for example, the state of California rewarded teachers whose students made significant test-score improvements with bonuses of $25,000.
If, on the other hand, a teacher’s students test poorly, the teacher may be censured or passed over for a pay increase or promotion. If an entire school tests poorly, federal funding can be withheld, the school can be placed on probation, and teachers can lose their jobs.
Even before the NCLB law was enacted, many states administered annual standardized tests to elementary and secondary school students, and some states sanctioned schools that didn’t do well. The advent of high-stakes testing laws, however, has radically changed the incentives for teachers to cheat.
Teachers cheat in Chicago
It’s actually pretty easy for teachers to cheat on standardized tests administered to students. They might, for example, simply give the answers to students, as one brazen fifth-grade teacher did in Oakland, according to Levitt and Dubner. They might allow extra time for testing, prepare students ahead of time for specific questions, teach testing techniques to the exclusion of worthier curricular pursuits, or simply fill in blank answers or correct wrong answers before turning test sheets in for electronic scanning. This last technique — the artificial boosting of classroom achievement scores by altering students’ answer sheets — was the one identified by Levitt.
From a database of nearly 100 million test answers
Levitt analyzed a database provided by the Chicago Public School system of nearly 100 million individual test answers for every student from third grade through seventh grade for the years 1993 to 2000. He found, for example, that in one sixth-grade classroom 15 students out of 22 all managed to reel off the same consecutive string of six correct answers on their test sheets. The string of six questions occurred near the end of the test, where questions tend to be the most difficult. Several of the 15 students who answered the series of six questions correctly left at least four questions in the same section blank, indicating that they would probably not have been able to answer the string of six questions correctly on their own.
Levitt found evidence of cheating in more than 200 classrooms per year
Levitt’s algorithmic analysis was able to identify only the most extreme cases of cheating — the cases in which teachers systematically changed students’ answers — and not the many subtler ways a teacher might cheat. Even so, his analysis revealed evidence of cheating in more than 200 classrooms per year, about 5% of the total. Tellingly, the sixth-graders described in the example above performed poorly in fifth grade, made nearly twice the expected improvement in sixth grade, and returned to their previous low-performance levels in seventh grade. Levitt’s analysis also showed a pronounced spike in cheating in 1996, the year when the Chicago Public School system introduced high-stakes testing.
Teachers cheat in Atlanta, too
A recent statewide review of standardized tests taken by school children suggests widespread cheating by teachers in Georgia, too. In an online article, the Economist reports that cheating may have occurred in 191 elementary and middle schools, including 69% of such schools in Atlanta. At one middle school in southwestern Atlanta, nearly 90% of the classrooms had suspicious test results.
Thirteen times more erasures than average
Following Levitt’s lead, the Georgia review looked for unusually high numbers of test sheet erasures involving a wrong answer changed to a right one. The number of wrong-to-right changes in the classrooms suspected of cheating were 13 times higher than average. The company analyzing the test sheets flagged those classrooms whose number of wrong-to-right erasures exceeded three standard deviations from the mean. The probability of that many erasures happening without manipulation is one-tenth of one per cent.
Resulted in 13 lashes
Georgia’s experience with high-stakes testing evidences a predictable trend. In 2005, 83%, 80% and 68% of the state’s eighth-grade students passed reading, language-arts, and math tests, respectively. By 2009, those percentages had risen to 93, 92 and 70. In Atlanta, scores in all grades and subjects rose continuously between the years 2000 and 2008. In 2009, after years of solid improvement in the test scores of Georgia’s public schools, 13 teachers, including the principal and assistant principal at one elementary school, were punished for cheating.
And a mixed bag of incentives for 2010
The effects on test scores of the punishments dished out in Georgia are yet to be seen. In economic terms, teachers — along with sumo wrestlers, drug dealers, parents, and everyone else — respond to incentives, negative and positive, moral, economic, and social. Incentives in the high-stakes testing environment of the Georgia public schools, where teachers now know that cheating is likely to have the same consequences as poor student performance, are a very mixed bag.