Writing Score Change Worth about 11 points

Now that I'm hanging out in the Ivory Tower with college kids I don't run into my old colleagues much any more. So it was great to see Phillip Shepard and Michael Miller, a couple of Fayette County refugees now working in the big house at KDE. It was also good to say "Hi" to Susan Weston Perkins, Roger Marcum, Dale Brown and Dick Innes. Yes, we smile and shake hands when we meet.

I missed greeting Bob Sexton and Cindy Heine, but got to meet Sharon Oxendine.

Thanks to Elaine Farris for the hospitality and to Harry Moberley for the shout out to my class during the meeting. And thanks to Moberley and Dan Kelly who have agreed to chat politics and social policy with EKU's doc students this fall.

Susan Weston Perkins commented about KSN&C's prior post on the inflation of high school writing scores when the scoring system was adjusted this year. With the disclaimer that she hasn't talked with Ken Draut about it yet, here's her take:

PERFORMANCE LEVEL CHANGE

Depending on the subject, student performance may be assessed to be

• At one of five performance levels: non-performance, novice, apprentice, proficient, or distinguished.
• At one of eight performance levels: non-performance, medium novice, high novice, low apprentice, medium apprentice, high apprentice, proficient, or
  distinguished.

From 1999-2006, writing was assessed at the five levels.

In 2007, elementary and middle school writing switched to eight levels, but high school stayed at five levels.

In 2008, high school switched to eight.

THE INDEX CHANGE

The Writing Portfolio Index on a 0-140 scale is calculated by multiplying the percent of students at each level by a weight.

In the five-level version, all apprentices are worth 0.60, but in the eight-level version, low is worth 0. 40, medium 0.60, and high 0.80.

So, if a school had a lot of high apprentices, they’d be better off with the eight-point scale. If a lot of low apprentices, they might prefer the five-point version.

There’s a similar issue with the novice scoring.

As it happens, the 2008 high school writing scores included [a] high number of high apprentices and high novices—so the switch to eight levels gave us a an impressive statewide jump...

(Apologies for the poor quality scan, KSN&C)

ROUGH IMPACT

From 2007 (five-level) to 2008 (eight level), the high school writing index went from a 56 to a 72.

It’s possible to go back and use the five-level formula. You just weight all apprentices at .60 and treat both medium and high novices as .13. If the state had done that, the Index would be just a 61.

In other words, 11 point come from that giant weighting switch.

THE REMAINING PUZZLES

Why did they switch elementary and middle in 2007 and high in 2008? KDE clearly decided to move to the eight-level scoring some time ago. Why the delay?

Also, why did they present the 2007 and 2008 data as though it’s a continuous trend? They made all that effort to say you couldn’t show trendlines for similar changes from 2006 to 2007. Why didn’t they repeat that on this one 2007 to 2008 change?

I don’t know how Ken Draut would answer these questions and need to make
time to ask him.

Thanks Susan.