Do the Math

Christel House’s Grade Doesn’t Add Up

For those of you who like to dig past the rhetoric...

This from Anne Hyslop at the New American Foundation:

Mel Horowitz: You mean to tell me that you argued your way from a C+ to an A-? 

Cher: Totally based on my powers of persuasion, you proud? 

Mel Horowitz: Honey, I couldn't be happier than if they were based on real grades.

Turns out we’ve all been Clueless when it comes to Indiana’s A-F school grades. Former Indiana (and current Florida) schools’ chief Tony Bennett has been under fire for released emails that show he and officials at the Indiana Department of Education altered the grades for certain schools prior to the very-public release of the new accountability measures last fall. What’s particularly worrisome is that the change to the grading methodology wasn’t so public. In fact, it was never announced. And from the emails obtained by AP reporter Tom LoBianco, it’s clear that Christel House’s initial grade set off a firestorm of panic at the IN DOE.

In a press call and separate interview with AEI’s Rick Hess, Bennett explained the matter by saying that Christel House Academy and a dozen other schools were unfairly penalized due to their unconventional grade configurations. Because they didn’t serve students in grades 11 or 12, these schools were missing key data elements for the high school calculation – namely, graduation rates and college readiness indicators, which typically count for 40 percent of the high school model. In Bennett’s words:

“The backstory is simple here, Rick. In our first run of the new school calculations in Indiana, we turned up an anomaly in the results. As we were looking at the grades we were giving our schools, we realized that state law created an unfair penalty for schools that didn't have 11th and 12th grades. Statewide, there were 13 schools in question had unusual grade configurations. The data for grades 11 and 12 came in as zero. When we caught it, we fixed it. That's what this is all about…. Because Christel House was a K-10 school, the systems essentially counted the other two grades as zeroes. That brought the school's score down from an "A" to a "C".”

Turns out it’s not quite that simple. The state has several variations of its grading rubric to apply to different school situations and set-ups. The basic models are 1) elementary and/or middle school grades and 2) high school grades. Then, there is a combined model for schools that have students in grades preK-8 and grades 9-12 – like Christel House, which served students through 10^th grade in 2011-12. The grade point averages for the 3-8 portion of the school and the 9-12 portion of the school are weighted according to the percentage of enrolled students in each grade span to arrive at one final, combined grade. (The final scale: 3.51 – 4.00 points = A; 3.00 – 3.50 points = B; 2.00 – 2.99 points = C; 1.00 – 1.99 points = D; 0.00 – 0.99 points = F)

Within the two basic models (ES/MS and HS), there are also deviations for special circumstances. Typically high school grades are calculated with a 60% weight on proficiency in end-of-course exams in Algebra I and English 10 (with potential bonus points for increases in proficiency rates from grades 8-10 and grades 10-12), 30% weight on graduation rates, and 10% weight on college readiness indicators. But some high schools are given special consideration: small schools, HS feeder schools (grade 9 only), 9-10 schools, and 11-12 schools. In the 9-10 model, proficiency rates make up the entire school grade, split evenly between Algebra I and English 10, and the bonus points do not apply.

Confused yet? Bear with me. Christel House should have been evaluated using a mixture of two of the models: the 9-10 model and the combined ES/MS + HS model. Except they weren’t. Because Christel House wouldn’t have gotten an ‘A’ that way. In fact, one of the released emails walks through the calculation (using preliminary, rather than final, achievement data). Under this method, Christel House earned a ‘C’ grade, “a HUGE problem for us” according to officials. And it set off the panic within the Indiana Department of Education – at 2:30 in the morning on September 13.

However, state officials soon – that same day, in fact – came upon a solution. Or in their words, a “loophole,” in the combined model calculation. Here’s the original definition (as written in one of the emails):

(j) A school’s… grade shall be determined by:
(i) Multiplying the average of the ELA and Math points for the EMS grades by the percentage of all students
(ii) Multiplying the sum of the four weighted scores for the high school by the percentage of students.”

Those three bold words contain the loophole Will Krebs, then Director of Policy and Research, found later that day – dubbed “option one.” Because Christel House didn’t have four weighted scores for its high school, the argument was that the combined school methodology was invalid. Without graduation rates and college readiness indicators, the school only had two of the four weighted components. Jon Gubera, Chief Accountability Officer, signed off on this option the following morning writing, “Option one works…. This would eliminate the HS points and ensure Christel House receives at least a B.”

So what does that mean, exactly? In truth, Christel House was never evaluated on its poor high school performance. Instead, all of the high school data were thrown out – a little detail Bennett failed to mention. Christel House’s ‘A’ is based on the ES/MS model only. As you can see below, Christel House’s grade was clearly inflated. The initial data run showed the school with a ‘C’ grade. Using the combined methodology sans “loophole” with its final performance data, however, the school would have actually earned a ‘B.’ Yet the school still received an ‘A’ from the state and was treated as only having elementary and middle school grades. Further, there is no indication anywhere on the state’s school report card that Christel House’s grade fails to reflect the school’s poor high school math performance.

According to the Indianapolis Star, Bennett refused to allow two regular public schools facing state takeover to use a similar "loophole" a year earlier. In both cases, poor middle school performance (where the school had recently expanded) penalized the high school. If their grades could not be separated, why was Bennett so eager to make an exception for Christel House?

These kinds of shenanigans are unacceptable and have chipped away at public faith in the legitimacy of school accountability systems over the last 10+ years of No Child Left Behind. Christel House’s grade is simply more false advertising from states and local districts that have a long history of finding loopholes in accountability systems and exploiting them. In fact, Indiana officials questioned whether using the loophole in this case would encourage other schools to adopt a grade 6-10 model to avoid accountability. Gubera replied: “Not in the immediate if we don’t advertise this everywhere.”

This just illustrates the problem. Christel House is an ‘A’ school… but only for its elementary and middle school program. Yet that isn’t the story Bennett and his staff are telling. This grade inflation is particularly unfortunate in Indiana, where parents and families have a greater degree of school choice than in most states and rely on information like A-F grades to determine where to enroll their children.
The thing is, Tony Bennett knows this:

“This kind of system has to make sense for the end user, in this case, the family… Back in Indiana, we were trying to build a new system. It's an interesting parallel. My recommendation to the Florida board was, "If your system doesn't fully make sense, then how do you defend it?" If the results come out suspect, then, in the end, you can really question the integrity of the system.”

Commissioner Bennett, Christel House’s inflated grade is suspect, and I’m questioning the integrity of the system. Accountability systems – even those required from the U.S. Department of Education – can be done right, but Tony Bennett unfortunately just made it that much harder to make the case for them.

Note: To see option 1 in action for yourself, check out the attached speadsheet from Indiana's Office of Accountability. Christel House Academy appears on the Elementary/Middle School tab, but not on the High School or Combined School tabs.

Hat tip to Dana Goldstein.