Review Finds Studies of Charter Schools Flawed, Problematic

This from State EdWatch:

Most studies of charter schools use unsophisticated methods and are flawed in ways that prevent researchers from accurately gauging those institutions' impact on student achievement, a new review concludes.

And while researchers have options for collecting more accurate information about charter school performance, they also face obstacles along the way—some of them related to the unwillingness among states and schools to provide crucial data, the analysis finds.

A meta-analysis of charter school studies revealed that about 75 percent of them do not meet rigorous research standards because they don't account for the differences in academic background and academic histories of students attending charters, when comparing them with those attending traditional public schools, according to the review, published in the renowned journal Science. Those studies typically fail to "disentangle school quality from the preexisting achievement level," or student self-selection of schools, the article says.

The article was written by Julian R. Betts, a professor of economics at the University of California, San Diego, and Richard C. Atkinson, a former president of the University of California system who also once served as director of the National Science Foundation.

High-quality research on charters is nonetheless beginning to emerge, they say. Much of it is coming from charters that have so many applicants that they must use lotteries for admission. Because getting in or not getting in is based on chance, the students who fail to secure a spot represent a sound "control" group necessary for a study, Betts and Atkinson say.

The relatively small number of lottery-based studies of charter schools have generally shown that they either outperform or perform at the same level as traditional public schools, according to the authors. But those studies cover only a small fraction—about 2 percent—of charter schools nationally.

Of course, many education advocates and scholars have long been frustrated by the paucity of rigorous research across the spectrum of school policy—not just with charters. (See my colleague Debbie Viadero's exceptional reporting on this.) The shortage of research is sometimes attributed to the difficulty of arranging high-quality, sustained studies of student populations in school settings...