Charter School Researchers Spar over Analyses

This from Education Week by way of KSBA:

The authors of a recent national study that found students in regular public schools outperforming their charter school peers are rebutting criticism that their research suffered from a “serious mathematical mistake” that negatively skewed the results.

In a memorandum posted online today, researchers from the Center for Research on Education Outcomes at Stanford University respond to criticism made last month by their Stanford colleague Caroline M. Hoxby. Ms. Hoxby, an economics professor, issued a memo critiquing the CREDO study in tandem with results from her own study of charter schools in New York City.

That study showed that charter schools in the city were having the opposite effect on their students’ achievement as the CREDO researchers found. Compared to their public school counterparts, Ms. Hoxby found, students in charter schools made more
progress in closing achievement gaps with better-off peers in suburbia.

“She claims that we modeled charter schools in a particular way and that was inaccurate,” Margaret E. Raymond, CREDO’s director, said of Ms. Hoxby’s
critique of the statistical analysis. “And her proof was based on her inaccurate specification.”

Ms. Raymond said Ms. Hoxby’s critique also failed to consider the many factors that the CREDO study took into account in its analysis.

“Imagine that we’re going to compare height and shoe size, and that we’re going to do it for all the people in the office,” she said. “Those two things might actually correlate in a particular way because they’re only two factors. But, if we’re going to predict shoe size based on many more factors, like age or parents’ height or parents’ shoe sizes, that’s a much more precise way of estimating.”

The CREDO researchers also argue in their memorandum that Ms. Hoxby’s estimates of the magnitude of the bias in their results was faulty.

The researchers’ arguments failed, however, to persuade Ms. Hoxby. In an e-mail response to Education Week, she said CREDO’s analysis contained “logically incorrect assertions.”

“They assert that the proof proceeds in a way that it does not,” she writes. “Since their main rebuttal is based on misreading a simple proof, their rebuttal is wrong.”

If the debate is taking on a bitter tone, it’s because the stakes are high. U.S. Secretary of Education Arne Duncan has been pressuring states to remove caps on the growth of charter schools in order to qualify for some of the $100 billion in economic-stimulus funds targeted to education. But opposition to the schools, which are public schools that operate largely free of the usual bureaucratic rules, continues to come from teachers’ unions and other education leaders.