Tuesday, May 01, 2012

Link between value-added ratings and classroom observations is modest at best, studies show

Here’s a statistical postscript to the story of Sarah Wysocki , the MacFarland Middle School teacher fired by DCPS last year after she earned good classroom observation scores on IMPACT’s Teaching and Learning Framework, but low value-added marks on student test results.

Ideally, there would be a healthy statistical link between the value- added and TLF scores. That would provide some solid evidence that following DCPS prescriptives in the classroom leads to higher student achievement on standardized tests.

“Nationally the hope is that there is a strong correlation between a teacher’s score on an instructional rubric and his or her value-added score,” Rachel Curtis, former assistant superintendent in the Boston Public Schools, wrote in her 2011 study of IMPACT for the Aspen Institute. “This would validate the instructional rubric by showing that doing well in instruction produces better student outcomes.”

So far, after two full years under IMPACT, the correlation is modest at best. An analysis of scores by DCPS consultant Mathematica shows that it was 0.34 (+1 indicating the highest possible positive correlation, and 0 indicating no correlation) in 2009-10. Last year it was 0.35.

Jason Kamras, DCPS human capital chief, said two years of data is not enough to draw strong conclusions. The correlation was comparable, and in some cases a little higher, than in other studies. For Cincinnati’s value-added system, which covers the same grades as D.C., a study showed a three-year correlation of 0.35 in reading and 0.33 in math. A study by the Gates Foundation called Measures of Effective Teaching, while less comparable methodologically, found correlations of 0.12 to 0.34 in math and 0.09 to 0.24 in reading.

“Would we like to see it higher? Yes,” Kamras said. “The reality is that these things are complicated and nuanced. They’re not measuring the exact same things. One is a measurement of the practice, the other is the product of that practice.”

Kamras, whose team is working on a series of changes to IMPACT to be announced sometime this summer, said the low correlation “is something we continue to wrestle with and question.”


Anonymous said...

I am sure all those effective teachers (who served real live children and were evaluated by a real live human principal) but now find themselves in the unemployment thanks to IMPACT are down right comforted to know that Kamras and his crew "continue to wrestle with and question" the low corrolation.

Teaching is a human business, not some darn physics formula (no offense to physics teachers).

Does this guy even hear what he is saying - "these things are complicated and nuanced....they are not measuring the exact same things.???

You have a teacher who is evaluated by a principal as an effective instructor and you may very well even have kids whose scores are above average, but you are going to fire them because some instrument which supposedly measures value added growth indicates less than a years worth of growth.

This is crazy, eventually no one will be in the classroom and we will religate our childrens' education to an IPad

Pearl_Anne2012 said...

In terms of teaching she wanted to provide the best education possible and go over and beyond the call of duty to make that a reality, without excuses.

classroom observation resource