Jack Jennings on the new CEP Study

KSN&C recently posted a story on the new report from the Center on Education Policy, "Has Student Achievement Increased Since 2002?"

CEP presents what is at least the fourth study in the past two years to confirm the significant progress being made in Kentucky schools - one of only four states to show moderate-to-large gains in both reading and math.

Richard Innes, a Bluegrass Institute researcher, doubted the credibility of the study.

Last year, a Kentucky Long-Term Policy Research Center study revealed that Kentucky made across-the-board improvements in its rank among the states; growing from roughly from 43rd to the 34th.

The Bluegrass Institute doubted the credibility of that study, too.

The KLTPRC report mirrored two previous studies done outside of Kentucky. Kentucky was 34th in Education Week's Quality Counts 2007 Achievement Index and was 31st in the Morgan Quinto 2006-2007 Smartest State Index."

I assume BGI didn't think much of those studies either.

When Richard questioned the quality of this study, I contacted Jack Jennings, President of CEP for a comment.

Here's his response:

Mr. Day, you sent a message on Wednesday saying that a commentator had a question about the report we released on Tuesday about student achievement. In
particular, he claimed our report would have credibility problems because Kentucky had a decline in 8th grade reading on NAEP that had not been reflected in the results on the state’s test. I asked the authors of the report to look at this issue and to respond. This is what they say:

First, this is consistent with the CEP report. Table 4-F (in the national summary) shows a moderate-to-large gain on the state assessment but a moderate-to-large
decrease on NAEP for grade 8 reading.

In the state profile, the first bullet says: "From 2002 to 2006, student achievement increased in both reading and math at all grade levels analyzed, according to percentages proficient and effect sizes." This is true for the state assessment, and presumably this is what the e-mail author is referring to.

Kentucky changed its assessment in 2007, so it's possible (although admittedly quite a stretch) that performance dropped in 2007... which showed up on NAEP but we couldn't see on state testing.

The report provides several reasons why NAEP and state tests might differ.

After receiving your e-mail, I got a call from a person from an organization in Kentucky who was asking the same question about our report. I asked him if he had read the report and he said that he had not, and that he was probably the source of your question. I urged him to read the report before he started raising concerns about it.

By the way, the report—Has Student Achievement Increased Since 2002?—and the fifty state profiles, as well as all the test score data from the states is on our web site, CEP-DC.org.

Thanks.

Jack Jennings, President, Center on Education Policy

Now, Innes's suspicions were not irrational. Knowing that there was a decline in 8th grade reading on the NAEP seems inconsistent with the study's conclusions and knowing that the researchers accounted for the NAEP data is important.

But taken as a whole, BGI tends only to report the bad news about public schools...and I suspect he's was looking to discount the good news about Kentucky schools any time he can. It's probably in his job description.

It would seem any study that produces what BGI believes to be "the wrong answer" must necessarily be a bad study. This is a problem whenever one is hired to substantiate their boss's predetermine perception of reality.