Has the CATS Assessment been reduced to a "Tall Tale?"

Bluegrass Institute analyst, Dick Innes, reports KDE's

"new CATS scoring is being adjusted by – a norming process! The CATS’ final score adjustment is very similar to that used in those very same nationally normed tests that the KDE consistently told us were substandard quality."

There's nothing inherently wrong with norming test scores - if - you tell folks that's what it is.

If KDE is telling schools that scores will regress toward the mean...isn't that a lot like...closing the gap statistically? The top schools scores are depressed, the lowest enhanced...

The CATS test has now gotten pretty far away from the original concept of a perfromance-based assessment as described by KDE under KERA post 1990. It is unclear if the Kentucky Board of Education is redefining "performance-based"; or not, This would seem to be a good time for the board of education to send out a clear message. Where is assessment in Kentucky heading?

Are we stretching the limits of sound assessment theory?

I like the idea of a n independent group to report assessment data to the state.

Here's my plan: I'm calling for the development of the Association of Assessment Gamesmanship Prevention. I want Ben Oldham, Tom Guskey and Skip Kifer to run it. They can flip a coin to see who is in charge: (Assuming 5 trials of 50 coin tosses each: Record the number of times the coin lands heads up. The statistician whose frequency of "heads up" events shows the greatest deviation from the mean, whether positive or negative, over the 5 trials, shall be declared Grand Statistical Poobah and will run the show.)

Poor design trumps good intentions. We'd better know where we're going.

Innes ascribes motive:

Absent significant progress, which is notably absent in credible measures like the NAEP, the KDE clearly needed to inflate scores. But, I think it would have been too obvious if the KDE inflated the overall scoring standards in a way that also inflated the final accountability score for all the schools.

Thus, to hide what was happening, the KDE needed a scheme that didn’t inflate everyone’s performance but that did boost overall scores in a way that made the state’s average look better.

In particular, the KDE benefits most by boosting the apparent performance of the lowest performing schools due to the way the CATS is actually scored. If more of those schools could be boosted into the CATS “Progressing Zone,” the schools wouldn’t face any sanctions and would appear to be doing better even if they didn’t make the goal. That would make the department look better than having performance spread out so that more schools wound up in the failing category while more schools also wound up topping out their CATS scores. What the department needed was a scheme that would make all the scores appear closer together.

Out on a Limb

It won't be long now. In a few more hours, CATS data will become public.

I have predicted jumps somewhere around 7 points will result...based on scant hard data. I am expecting dips at the top, leaps at the bottom. An overall swhift upward.

The over and under on Booker T Washington is 10; Harrison is 5; Fayette County is 8; The state is 7 ...

Why on Earth would anyone bother?

Well...some of it is sport. Over the years, it was my practice to predict test results; my own and others. It's a principal thing. If your numbers are good...it's fun.

Some of it is because it's important.

Tomorrow morning: Will it be scholar or dunce?