This week, New York City canceled a three-year-old test program that offered cash bonuses to teachers in the hopes it would help boost their students’ test scores after a new study found that the program had no such effect.

The Schoolwide Performance Bonus Plan was initiated in the 2007-2008 school year as a pilot program between the New York City Department of Education and the teachers union UFT, the United Federation of Teachers. In January the Department of Education suspended the program after several years of unstable dips and peaks in test scores. The most recent drop in scores occurred after New York raised state standards.

Schools that met test score targets that were agreed upon with the district would receive $3,000 per union member in the school. The program, which was initiated in schools located in poor neighborhoods, was only initiated with union members who first agreed to participate. The effort, which gave away tens of millions of dollars in its inaugural years, gave away just $4.2 million last year because of plummeting test scores.

The study, conducted by the research group RAND, found that the teacher bonus program didn’t in fact have any effect on schoolwide performance, or on student achievement in any grade level. It was initiated in elementary, middle and K-8 schools, and at none of these grade levels did students in the program do better in any year on standardized tests than their peers in a control group.

A 2010 study from Vanderbilt University found similar results with another pay-for-performance model. The reports have given ammunition to critics of accountability measures that demand that teachers’ job evaluations and their job security be tied to their students’ test score performance. Such reforms are unrealistic, critics say, and do not take into account the myriad factors that influence a child’s education, such as poverty, or their home life, or the educational inequalities children from poor neighborhoods face.