Link between value-added ratings and classroom observations is modest at best, studies show

This from Bill Turque at The Washington Post:

Here’s a statistical postscript to the story of Sarah Wysocki , the MacFarland Middle School teacher fired by DCPS last year after she earned good classroom observation scores on IMPACT’s Teaching and Learning Framework, but low value-added marks on student test results.

Ideally, there would be a healthy statistical link between the value- added and TLF scores. That would provide some solid evidence that following DCPS prescriptives in the classroom leads to higher student achievement on standardized tests.

“Nationally the hope is that there is a strong correlation between a teacher’s score on an instructional rubric and his or her value-added score,” Rachel Curtis, former assistant superintendent in the Boston Public Schools, wrote in her 2011 study of IMPACT for the Aspen Institute. “This would validate the instructional rubric by showing that doing well in instruction produces better student outcomes.”

So far, after two full years under IMPACT, the correlation is modest at best. An analysis of scores by DCPS consultant Mathematica shows that it was 0.34 (+1 indicating the highest possible positive correlation, and 0 indicating no correlation) in 2009-10. Last year it was 0.35.

Jason Kamras, DCPS human capital chief, said two years of data is not enough to draw strong conclusions. The correlation was comparable, and in some cases a little higher, than in other studies. For Cincinnati’s value-added system, which covers the same grades as D.C., a study showed a three-year correlation of 0.35 in reading and 0.33 in math. A study by the Gates Foundation called Measures of Effective Teaching, while less comparable methodologically, found correlations of 0.12 to 0.34 in math and 0.09 to 0.24 in reading.

“Would we like to see it higher? Yes,” Kamras said. “The reality is that these things are complicated and nuanced. They’re not measuring the exact same things. One is a measurement of the practice, the other is the product of that practice.”

Kamras, whose team is working on a series of changes to IMPACT to be announced sometime this summer, said the low correlation “is something we continue to wrestle with and question.”