Friday, June 01, 2007

Test score inflation: Coming soon to a state near you

One of Kentucky Education Commissioner Barbara Erwin's first duties will be to announce remarkable gains made by Kentucky students as measured by state tests.
The announcement will be made this coming fall.


The tests, recently completed, have not been scored. No reports are out. I have no scientific studies to back me up. No official statements have been issued by KDE that support my...let's call it a hunch.

And perhaps, I'm jumping the gun here, but I find it curious that I still haven't been able to find a teacher or principal - not one person - who thinks this year's test was as hard as past assessments. And I've been trying. The best I can offer is one 10th grade social studies teacher who thinks the test is about as difficult as before.


One elementary academic coach (in a tough school) actually laughed at my inquiry, giving me a big Cheshire Cat grin that only comes with that certain feeling of future success, and job security.

But all around central Kentucky, teachers and principals are smiling sheepishly because they believe the test they just gave to Kentucky students is noticably easier than past assessments.

Something's going on here and the most likely explanation is that Kentucky has (silently) joined the group of states that have dumbed down the test in order to realize numerical gains. Nationally—in response to NCLB—states have been tinkering with their state exams, in part, to help boost pass rates.

It makes for a better press conference.

It is almost comical that - if my hunch proves true - once again, Barbara Erwin will benefit - just as she did in Illinois last year.

The "benefit" comes from her ability to claim sole responsibility for impressive student achievement gains as though they were a result of her action. In reality, big gains were seen across Illinois only AFTER significant changes were made to the assessment.

In March, the Chicago Tribune reported, "Why test scores went up."

Illinois elementary school pupils passed the newly revamped state achievement exams at record rates last year, but critics suggest it was more the result of changes to the tests than real progress by pupils.

Overall, children passed 77 percent of all math, reading and science exams they took in 2006, compared with 69 percent the year before, according to a Tribune analysis of test data.

The 8 percentage-point gain is the biggest one-year increase since the state started giving the Illinois Standards Achievement Test in 1999.On most exams, the 2005-2006 gains outpaced the improvement made over the previous five years combined.
Low-income and minority pupils posted the largest increases, which helped narrow the pernicious gap between their performance and that of their white, Asian and more affluent counterparts.

...Most notably, the state dramatically lowered the passing bar on the 8th-grade math test. As a result—after hovering at about 50 percent for five years—the pass rate shot up to 78 percent last year.

...Pupils also were scored on one extended response, compared with two the year before. Extended response, which requires pupils to answer in essay form, is tougher than multiple choice.

"...It doesn't take a rocket scientist to see that something doesn't compute here," said Lawrence Aleamoni, an education professor at the University of Arizona who has written extensively about tests. "The big jumps tell me something is going on here. All of the changes they made to the test will certainly inflate the scores."

But state education officials argue that the 2006 exams are just as difficult as the previous tests."Everyone on our staff wholeheartedly believes that teachers prepared their kids well," said Becky McCabe, who oversees testing for the Illinois State Board of Education, which administers the tests. "The kids were prepared for the tests and it shows in the results."

Statistically, the lower the pass rate, the easier it is to show progress. It's tougher to move from 80 to 90 percent passing, than from 30 to 40 percent. In Illinois, the test changes helped lift the scores of all but 52 of 766 elementary and unit school districts. The pass rate of low-income pupils on the 8th-grade math exam doubled, from 32 percent in 2005 to 64 percent last year.

"It's an anomaly that blows the credibility of this test," said Jeff Mays, president of the Illinois Business RoundTable, which opposed changes to the 8th-grade math test. "I think we need an independent party to come and look at this and help us build a road map to credibility."


From another Chicago Tribune piece: "There is clearly a race to the bottom going on," said Kevin Carey, a policy director at Education Sector, a think tank that studied state testing changes. "When states change rules under No Child Left Behind, it's always changes that will make it easier for schools. One state will come up with an `innovative' way to give schools the statistical benefit of the doubt and then every state will follow suit."

Education Sector's analysis from Thomas Toch's "Flunking Tests:"

Testing industry experts say that many of the tests that states are introducing under NCLB contain mostly questions that require students merely to recall or restate facts rather than doing more demanding tasks like applying or evaluating information, largely because it's easier and cheaper to test the simpler tasks.

Experts say that "open-ended" questions, which require students to produce their own answers rather than select from among answers supplied by test-writers (the format of multiple-choice questions), are a good way to gauge students' grasp of higher-level skills....Low-level questions have a role; it's important that students grasp the most basic skills.

But NCLB has sought to lift the level of teaching in the nation's classrooms by requiring states to set challenging standards. Tests that demand students learn only low-level skills give teachers incentives to drive down the level of instruction in their classrooms.

They also give an inflated sense of student achievement.

Scores on reading tests that measure mainly literal comprehension are going to be higher than those on tests with a lot of questions that require students to evaluate what they've read by, say, reading two passages and identifying themes common to both. The same is true in math.

That's how it was in Illinois when Erwin's District 303 posted what would have otherwise been impressive gains.

I'm guessing that's how it will be in Kentucky. But here, too, testing officials will ignore the obvious and claim that the gains are real. What else can they do?

But you can credit Barbara Erwin...if you want to.

No comments: