How do the schools really stack up?

Over at School Finance 101 Bruce Baker is doing his year end review of education data on the New Jersey schools. But his review contains some news that applies to Kentucky and all other states as well. I have added a few bits to highlight Kentucky's performance.

This from School Finance 101:

On many occasions I’ve pointed out better and worse uses of national and international assessment data – of Mis-NAEPery and PISA-Palooza… wherein the media and punditocracy go wild with gross misrepresentations and misinterpretations of relatively limited albeit not entirely useless assessment data.

In New Jersey, we’ve been told in recent years that while our average scores remain high, we must not rest on our laurels, because our gains pale by comparison to reformy standout states like Tennessee. We’ve been told that while our average performance is high, our gaps in achievement are among the largest in the nation and certainly not improving at any reasonable rate. We’ve also been told that these findings provide strong proof that all the money New Jersey has thrown at schools in response to years of litigation over school funding has not only been unhelpful, but that the additional funding to high poverty settings has actually caused harm. As such, the way to repair that harm is to reduce funding to high need settings and redistribute those harmful resources across other, less needy districts likely to use it more wisely (yeah… really… they did say that… and they’ve followed through on that redistribution plan!) And that will help fix the achievement gap!

But what really is the state of student outcomes in New Jersey, if we apply a few basic principles to the analysis of NAEP data – guidelines I have addressed in numerous previous posts:

• First, average state contexts differ and those differences strongly influence average NAEP scores at all grade levels. In short, poverty matters! As such, performance levels should be adjusted for poverty.
• Second, NAEP gains over time (which are cohort gains), are strongly influenced by initial NAEP performance. That is, those who started lower and had more to gain, gained more. As such, changes over time should be adjusted for initial scale scores.
• Third, achievement gaps between low income and non-low -income, or among races are substantially influenced by the income gaps between these groups. As such, achievement gaps should be adjusted for differences in income between groups.

Average Performance Level Adjusted for Poverty

This first figure shows us the scatterplot of state average poverty rate and average 8^th grade NAEP scale scores. Those states falling above the line have greater than expected scale scores and below the line have lower than expected scale scores. Notably, the correlations are quite strong. New Jersey beats the odds on both 8^th grade reading and math. That is, NJ scores are higher than would be expected even given New Jersey’s low poverty rate?

Figure 1

If we rank states by their average difference from expectations, New Jersey comes in fifth (averaging the math and reading differentials). [Kentucky comes in 7th!]

Figure 2

But that’s only because the whole U.S. stinks, right?

Of course, some might argue that its really nothing to cheer about – doing better than other U.S. states, because of course, we all know that the U.S. performs miserably compared to other nations. But as I’ve pointed out previously, when conducting similar poverty adjusted comparisons across countries, the U.S. doesn’t look so bad. (see here for more explanation/discussion)

Figure 3

Ah… but you say… outcomes of even high performing – non-low income kids in the U.S. still stink compared to those in other countries. Again, I respond by pointing out that most such comparisons are deeply conceptually flawed. Perhaps most importantly, as I’ve explained on numerous previous posts, the U.S. average is only as low as it is in international comparisons because of the large number of low performing (relatively high poverty) states that have largely thrown their education systems under the bus for the past several decades...

Here are a few more pictures...