Kentucky's Lowered CATS Standards Remained Relatively High

Over at Prichard, Susan Weston has been looking into the lowering of academic standards in Kentucky circa 2007.

You will recall, that is the time Kentucky was switching from the old CATS to the new (and not improved) CATS - thanks largely to NCLB's strictures screwing up a fairly good (yet, far from perfect) accountability system.

A retrospective report from the National Center for Education Statistics allows us to more clearly see the degree of lowering that actually went on. Turns out, Kentucky's reading and mathematics Kentucky Core Content Test (KCCT) standards were indeed lowered, but Kentucky still had higher proficiency standards than many other states.

The NCES report allows each state to compare its criteria for proficiency with that of other states and determine whether the rigor of its proficiency standards as represented by NAEP scale equivalents changed from 2005 to 2007. The analysis allows NAEP to corroborate state-reported changes in student achievement and provide states with an indicator of the construct validity and generalizability of their test results.

Weston reports here and here,

In fourth grade reading, the study shows:

• 206 as the NAEP score identified as equivalent to our "old CATS" proficiency standard.
• 205 as the NAEP score equivalent to our "new CATS proficiency standard.

In eight grade mathematics, the study shows:

• 285 as the NAEP score equivalent to "old CATS" proficiency.
• 279 as the NAEP score equivalent to "new CATS" proficiency.

In effect, the report gives us a formal demonstration that we lowered the proficiency bar in 2007. The reading change was quite small, but the math change more substantial.

The report also shows Kentucky's 2005 standards to have been quite high in national perspective, with only 21 percent of the included states having higher standards in either subject.

KSN&C argued against lowering academic standards at the time.

Kentucky to pursue federal education grant

Holliday hits town

Brothers re-elected Chair of KBE

Dorie Combs and C.B. Akins named vice chairs

This from C-J:

Kentucky's new education commissioner said Wednesday that his first priority is to win a competitive federal grant for school reform.

Commissioner Terry Holliday, in his first day on the job, told the state Board of Education that he plans to aggressively go after money through the $4.35 billion “Race to the Top Fund” unveiled by President Barack Obama late last month.

“The whole purpose is to get more kids graduating from high school and prepared for postsecondary work, whether it be community college, college or technical training, or even the armed services,” Holliday told reporters during a break in the monthly board meeting.

Money will be awarded to states that propose strategies for raising academic standards, adopting common student assessment tests and improving teacher quality.

School Board Chairman Joe Brothers said Kentucky is ahead of other states because of a bill passed by the legislature this year.

Senate Bill 1 scraps the Commonwealth Accountability Testing System — known as CATS — and requires the Department of Education to devise new educational standards and a new test for the 2011-12 school year.

“The word on the street, and in articles being written in education circles, is that Kentucky has positioned itself to reassume some of the leadership it had at one time and maybe has not sustained over time,” Brothers said...

Viewpoint: SB 1 and raising standards

Over at Prichard, Susan offers some observations on Senate Bill 1 and its opposition to Kentucky's lowered standards.

Since 1990, Kentucky testing has had three scoring systems, each easier than the last. By easier, I mean two things.

First, we set new "cut points" when we changed from 1998 KIRIS to 1999 CATS and when we changed from the 1999-2006 version of CATS to the revised 2007 and 2008 CATS. At the moment that we put the new cut points to work, many more
students were counted as proficient.

Second, we asked for less complex kinds of student performance. From KIRIS to CATS, we dropped performance events and added multiple choice. From "CATS I" to "CATS II" we gave multiple choice more weight than it had before.

Senate Bill 1 demands something different. It calls for P-12 standards to be aligned with college expectations...

CATS- Episode 2009

By Penney Sanders

Now that the KY General Assembly has adjourned, it may be safe to venture a few observations about what happened to the CATS assessment.

SB 1/HB508 reflected much needed updating and modifications to KY’s assessment system-CATS. It was not the end of school reform as we know it, contrary to the comments of some. Most school people, especially classroom teachers, have known for several years that there needed to be revisions to the test. It was cumbersome, took too much instructional time, did not provide useful data, etc, etc.

However, up until March 2009, the voices calling for changes to CATS had been effectively rebuffed no matter how reasonable the concerns expressed. In the 2008 Session, a previous iteration of SB1 was introduced. In some ways it was quite similar to the bill filed in this session. However, the 2008 version was met with tremendous opposition and the bill did not move forward. It was politicized, polarized and ultimately pulverized

What was the difference this year??? The changes in CATS legislation are an excellent example of the “Tipping Point”. As described in Malcolm Glidewell’s bestseller of the same name, there is a point at which an issue (legislation), despite previous resistance, changes; there becomes significant support for or opposition to that which was once thought unchangeable and suddenly it “tips.” There are other examples of tipping points-popularity, elections or public issues.

Criticism of CATS began in 1995 with the publication of the OEA’s report from their national panel of testing experts who identified problems with the assessment, then called KIRIS. Many of those problems could have been anticipated because testing for accountability was in its infancy and Kentucky was at the forefront of attempting such a broad-based test.

Over the years the changes to KIRIS/CATS at best, were band aids and at worst, exacerbated the weaknesses. The needed revisions and updates did not occur because of the prevailing and pervasive belief that changing the assessment somehow undermined school reform.

Now there is the opportunity to create a Kentucky test that reflects the best current thinking in psychometrics, revisit Kentucky’s instructional standards and implement a useful assessment.

Transforming the test to one based on individual student accountability and improvement from year to year reflects the current thinking in assessment and may prove to be a more accurate measurement of a school’s progress.

If significant numbers of students fail to make progress each year, then it is obvious something is wrong. On-demand writing may prove to be less cumbersome and time-consuming than portfolios.

Such a “Value Added” approach will encourage significant focus on what is occurring in the classroom and less on peripheral issues.

A second phenomenon was also present in the history of SB1/HB538 –the “argumentum ad homenum”-loosley translated it means attacking the man, not the argument. It is the lowest form of disputation. However, we have seen years of attacking the messenger rather than focusing on the merits of the message.

Throughout the 2009 session, until its last days, the voices that have been critical of CATS for several years were once again subjected to the attacks on their person. In my mind the low point, was the continued criticism and allegations from some that revising CATS was part of a right-wing attack on education led by those “rascal” Republicans. With the House vote of 93-0 and support from KEA to abolish CATS in its current form and to develop a new assessment, it is ridiculous to continue to pretend that somehow any opposition to testing must be rooted in ideology or political partisanship.

Kentucky has a wonderful opportunity over the next three years to create an assessment that reflects Kentucky’s academic standards. Additionally, it can be an assessment that is a useful diagnostic and instructional tool.

Furthermore, as we move forward to discuss and legislate other important issues, let us remember some of the lessons learned from KIRIS/CATS-that voices of criticism are not necessarily those of opposition but rather of different perspectives. As we undertake the comprehensive KERA review, the focus must be on the merits of the arguments rather than attributing motives and motivation to personalities and politics.

The Governor has created a unique opportunity to bring diverse voices and perspectives to this important discussion so that we can create the best possible education system. It is, hopefully, a new day and an opportunity that cannot be missed or misused.

Senate Bill 1 ends the CATS era. What will replace it?

By Skip Kifer

SB1 proposes a huge amount of testing. Some testing is statewide; other testing is not. Some tests are used for accountability purposes; others are not. Some tests are multiple- choice only; others are not. There are formative tests, summative tests, performance assessments, benchmark tests, interim tests, norm-referenced tests, criterion-referenced tests, end of course tests, and writing portfolios as well as program reviews and program audits which are used to measure students, schools and districts.

The Kentucky Department of Education (KDE) is directed to integrate the testing into a new assessment system. It has almost three years to complete the task.

I doubt that a defensible new system can be created, even with that amount of time, without additional resources for KDE and more focused testing.

A first step to building a new assessment is to find out whether what is presently being done works. Major components of the existing system should be evaluated. For instance, EPAs - Explore, Plan and ACT - produces results presumed to help students learn and school personnel make better decisions. Does it? What is the evidence that the information is used? What is the evidence that using the evidence makes a difference? On what? Studies of each major component should provide answers to those or similar questions. If the component is not working, it should be eliminated.

There should be a thorough understanding of the implications of moving from an assessment system focused on schools to one focused on students. CATS relied on different forms of the test to get a better sample of achievement within a school. A student might spend an hour taking a test but with six forms of the test, the assessment contained about six times as much information for each school. Even with that, about 25% of schools were misclassified. Some were said to be progressing when they were not. Others were said to be needing assistance when they were progressing.

The existing assessment had thousands of misclassifications at the student level, too. Proposing to produce longitudinal data on students places more pressure on the assessment and more obstacles to accurate measurement. Creating measures that can be scaled across grades suggests spending more time testing each student. Is that desirable? Is it practicable?

Finally, the biggie! After almost two decades of educational reform with a spotlight on accountability through high-stakes testing, is their evidence that such testing works?

Research results both in Kentucky and nationally are mixed. Test scores go up but there are questions of whether that is mainly because of teaching to the test. Because what is taught and tested narrows, higher test scores may not mean mastery of a content area and may mean that content areas not tested are not emphasized. In addition, states with high-stakes testing do no better, and perhaps worse, on national tests than states without such programs. Reasonable persons would agree that test scores are, at best, a narrow reflection of successful schools. Other aspects are more important. Evidence should be collected, therefore, that indicates whether after about two decades of high-stakes testing Kentucky schools:

a) are better places for kids than they were prior to the reform;
b) nurture talent in ways that it should be nurtured; and
c) insure those who emerge from the school system have commitments to democratic ideals, participation in democratic communities, and tolerance for varied persons and views.

Armed with good evidence, resources and time, I hope KDE creates a system that benefits the Commonwealth and its children.

FLUNKED

This form the Paducah Sun:

Few lament the quietus of CATS

Politics makes strange bedfellows, and when politics is intertwined with education reform, you never know who you might wake up beside.

Take CATS, the Commonwealth Accountability Testing System. On one side you’ll find the independent Prichard Committee for Academic Excellence and the editorial pages of Kentucky’s two largest newspapers.

On the other side you’ll find the liberal Kentucky Education Association, conservative public policy advocacy groups, every Democrat in the Kentucky Legislature, along with every Republican. Oh, and The Paducah Sun.

Both houses of the general assembly unanimously — unanimously! — backed Sen. Ken Winters’ bill to ditch CATS in Kentucky’s public schools. According to the Associated Press, both chambers erupted into applause after the vote.

The test could be more accurately called CUTS, for Commonwealth Unreliable Testing Sham. The ineffective test does less to measure student progress than give an illusion of progress whether genuine or not. It should have been deep-sixed long ago.

The legislation calls for replacing CATS, which includes open-response questions and writing portfolios, with a national norm-referenced test, still administered at the end of the school year. Scoring open-response and essay questions is necessarily subjective, making it impossible to accurately measure progress or compare Kentucky students’ scores with scores from other states. Replacing CATS is long overdue.

But the (Louisville) Courier-Journal called norm-referenced tests “outmoded, ineffective” in a blistering editorial Friday railing against “conservative, educationally revanchist Republicans,” aided by “lazy” Democrats and a backward Kentucky Education Association opposed to “new ideas.” The editorial lumped the state’s public teachers together, saying they are “grateful for being allowed to escape real accountability.”

Ouch. Sounds like the editorial writers in Louisville are not playing well with others.

The editorial even lamented “the tragic evisceration of (CATS) writing portfolio provisions.” Tragic? Hardly. Tragic describes a fatal car wreck or tornado, not a policy change.

Those writing portfolios, by the way, will be retained as a teaching tool in grades 5-12, as they should be, just not as part of the assessment.

The Lexington Herald-Leader echoed its rival paper Sunday, adding Gov. Beshear and House Speaker Greg Stumbo, both Democrats, to the list of bad actors, accusing them of caving to “the worst impulses of both the teachers union and conservative enemies of public schools.”

That’s called painting with a broad brush.

The self-evident escapes CATS’ defenders on the two editorial boards: For any testing instrument to be useful in tracking progress and comparing performance, it must use objective and measurable criteria. If anything is “outmoded,” it is the trendy educational philosophy behind subjective testing methods, designed to elevate self-esteem above actual achievement.

Ironically, those states that consistently exceed the national averages in norm-referenced tests don’t complain that the tests are ineffective.

We admit we don’t always, or even often, agree with the Kentucky Education Association. But they’re right this time. CATS was a time-consuming exercise in futility, yielding little information useful for tailoring education plans to address individual student deficiencies.

The KEA supports the planned CATS replacement. The legislation calls for a test to allow both the schools and parents to better track student performance. A side benefit is that it will help identify teachers whose classes consistently exceed expectations and those that regularly fall short. Administrators can prescribe remediation for under-performing teachers — peer coaching, additional coursework — just as teachers do for under-performing students.

Only a few will mourn the demise of CATS. The rest of us dinosaurs — teachers, legislators and proponents of public school accountability, whether liberal or conservative — are happy to say “good riddance” to CATS.

Figuring Out What Gets Tested

“If they’re not going to score it,

and if it’s not going to count for anything,

I don’t know why we should waste our time.”

Supt Rich Crowe, Frankfort

From district to district, and perhaps from school to school within districts, this spring's testing is certainly going to be different. Many districts will drop from having one too many accountability scores - down to having one too few. Where accountability scores exist at all, they will be calculated locally.

Fayette County Superintendent Stu Silberman seemed to get the ball rolling by declaring that Fayette County would proceed as usual by scoring writing portfolios, the Lexington Herald-Leader reported.

"The students already have put a lot of work into them, and we need to honor their commitment," Silberman said. "It's up to each individual school, but we've suggested that they complete the portfolios, grade them and give the students their grades and feedback."

Silberman also said the district expects to calculate and make public its own "academic index" based on this spring's testing, whether the state publishes a performance index or not.

On one hand, word about what and how the assessment is going to be different is still being filtered out to schools. According to KDE spokeswoman, Lisa Gross, so far the only guidance sent to districts from KDE is this chart. On the other hand, assessment time will be here before we know it. School folks are scrambling.

Adding to the confusion is the arts & humanities and practical living/vocational studies portions of the KCCT. As Gross confirmed, "the arts & humanities and practical living/vocational studies portions of the KCCT will still be sent to schools by our testing contractor, and they can administer them if they wish." Some teachers of these subjects have apparently been told that they can design their own tests from these questions.

This may have led other teachers to wonder if other sections of the test might be reconstructed by teachers - and they can not.

The Kentucky Board of Education meets next week. Additional guidance to school districts is likely to follow.

In Hopkins County, The Messenger (subscription) reports,

Schools' preparations for this year's state assessment are getting some last-minute revisions...."We got some inklings of it before we got the final version of Senate Bill 1," said Hopkins County Assistant Superintendent Linda Zellich. "We were pretty sure early on that things were going to change with the writing portfolio. What we're really surprised about is ... the timing issue, that it went into effect immediately."

Dan Evans, the district's assessment coordinator, said ..."If they want to revamp the testing system, certainly there are some things that need to be changed, but throwing things out that we've worked all year on makes no sense."

Zellich said she believes it will be the district's recommendation to score portfolios locally this year and provide feedback to students and teachers. The portfolios are "effort intensive" but "yield some good work," she said."There's no doubt ... that students' writing skills have improved," Zellich said...

..."It still remains very confusing to us," she said. "We're still not sure what we're going to do and when we're going to do it. That creates some anxiousness on our part because we want to know, our principals want to know and the public wants to know...

The Paducah Sun (subscription) reports that,

McCracken County students will be tested this spring in arts and humanities and practical living/vocational skills even though the legislature has attempted to change school testing... The McCracken school board passed a measure Thursday night allowing the district to continue testing on arts and humanities and practical living as well as the writing portfolios.

Paducah Assistant Superintendent Vickie Maley said she has talked with principals, and Paducah schools likely will score their own writing porfolios. As for the rest of the test, she said administrators are waiting to see the final outcome at the state level...

Eleanor Mills Spry, assistant superintendent for Murray Independent Schools, is attempting to stay positive about the changes. “One of the most positive aspects gathered from the legislation is that schools will have fewer days of assessment, which, in turn, provides additional instructional time for students.”

In northern Kentucky, Walton-Verona Schools Deputy Superintendent Gene Kirchner had to assuaged potential concerns about the district decreasing its focus on art and humanities.

"We will focus on the core skills that all children need to be successful (reading, writing, arithmetic) without sacrificing those things which create well rounded individuals (Science, Social Studies, Arts/Humanities, Practical Living/Vocational)," Kirchner said in the e-mail."We wanted our people to understand that our expectations haven't changed even though there's no state assessment (on the subjects)," Kirchner said.

In Hardin County,

Nannette Johnston, superintendent ... said once the bill is signed district officials will to decide what to do with the portions of the test that are discarded, such as the writing portfolio. Individual districts still can do their own assessments of these areas, and Johnston said she and others will decide what pieces will help them monitor student performance.

Jayne Risen Morgenthal, superintendent of Elizabethtown Independent Schools, said the quick turnaround between the possible changes and this year’s scheduled testing.
“I think the timing could have been better,” she said. ... Overall, Morgenthal is happy with the bill, but waiting for instructions has been difficult, she said.

In Pike County, schools Superintendent Roger Wagner is playing it safe. He notes that the legislature still has two days remaining in which to consider legislation. Should Governor Beshear veto this bill outright, or use line-item veto on the bill, it will go back to the legislature for review.

"Until he signs the bill, it is not law, and the current CATS testing laws stand
in place,"

Wagner said. Of course, SB 1 passed both houses without dissent and the Governor has said he will sign the bill.

“I can’t anticipate what the Governor says he’ll do,” Wagner said. “We’ll know
more at the end of the month.”calling, changes to testing law has created,
“a lot of confusion with everyone.”

SB 1 has gotten a mixed reception at Murray Independent and Calloway County schools.

Eleanor Spry, superintendent of curriculum and instruction for Murray Independent, said Wednesday that there are a lot of unknowns about the move so schools officials aren't planning any response until more information can be obtained.Spry said the district honors the intent of the CATS system and its predecessor and will continue to move toward a goal of academic achievement.“If instruction was important then, it still continues to be important whether you assess it or not” she said. “ So we value our teachers. We value the instruction they give and what they teach our kids.

Calloway County Schools Superintendent Steve Hoskins said the testing process is evolutionary“It's probably time to evolve now. It's advantageous to regroup. ... However Hoskins believes writing portfolios should continue to be an important part of the curriculum.

In Floyd County Superintendent Henry Webb sees the decision to modify the testing as a positive change

.“We are not going to see too many changes in the interim,” Webb said. “I think the changes are going to be beneficial for both students and staff. It’s a good move to eliminate the portfolios from the accountability process. I’m pleased with the changes that have been made, and I think it will continue to help us move forward.”

The State Journal reported last week that reformers' worst fears may already be coming true in some places.

Superintendent Rich Crowe told the Frankfort Independent Board of Education last week that he’s inclined to focus on curriculum areas that actually count toward the ratings. Scoring of writing portfolios may be scaled down. “If they’re not going to score it, and if it’s not going to count for anything, I don’t know why we should waste our time,” Crowe said.

Bill signals retreat from KERA goals

This from the Messenger-Inquirer:

Fourteen years ago, in early 1995, the mood was celebratory at Cravens Elementary School on Owensboro's west end. The little school that had failed to break 30 on the initial KIRIS accountability tests three years before, a score that reduced teachers to tears, was in line to receive the maximum financial reward for dramatic improvement on the battery of tests.

Cravens' score on the 1991-92 baseline test was 29.5. The next year the school improved to 35.6 on the test. The following year, fourth-graders recorded a score of 52.5, far exceeding the school's goal. Cravens, transformed by the gritty perseverance of its staff, was off and running.

Now we wonder if the days of that kind of success story, and many others like it, are over. While we would never doubt that many of our local educators will teach with every bit as much integrity as before, it's a fact that Kentucky is now in full retreat from the lofty goals set out by KERA in 1990. House and Senate members celebrated in Frankfort last week when the CATS tests were scrapped, and with it public school accountability. What was to have been a review of KERA and CATS, with adjustments being made where necessary, turned into a rout. Senate Bill 1, which Gov. Steve Beshear has indicated he'll sign, passed the Senate 38-0 and the House 93-0.

Just like that, with five years to go for schools to reach the goal of scoring 100 on the CATS test (several schools in this area are already there), the state will devise a new test and a new accountability system. Writing portfolios will no longer be part of the test. The testing period will be shortened. Multiple-choice questions will prevail, and open-response questions, which require students to actually write an answer to questions, will be downplayed. Arts and humanities and vocational studies will no longer be tested.

Schools that were in danger of not making their 2014 accountability goals can look forward to a new accountability system, bailed out, as it were by the legislature's drive to pull back from KERA's tough but fair demands...

...Cravens' faculty dug in deep. Principal Beverly McEnroe said teachers looked at every single thing they did and took everything apart. Every teacher was assigned to a committee to push for improvement in a different part of the Kentucky Education Reform Act. Teachers even started making home visits.

It is not our intention to single out Cravens, except to hold it up as an example of what was and remains possible in public education. Cravens' success was repeated in school after school in this area; it was indicative of what KERA accountability and the mantra that every student can learn and learn at a high level wrought. Better yet, these many years later, the success has continued.

Many of our local and area schools are performing wonderfully. The effort put forth, the expertise gained, the determination demonstrated during nearly two decades of the education reform and accountability movement in Kentucky in so many of our schools has been exemplary and often amazing.

We say this because we are immensely proud of principals, teachers and support personnel and everyone who embraced KERA and its monumental challenges. The good fight was fought, and a generation of students are better for it. In light of what happened last week, we worry about the next generation.

CATS Reaction Across the State

Senate Bill 1 could be signed into law today as across the state folks are coming to grips with what all this means.

"I believe there are several positive elements that could represent an important step forward in our ongoing efforts to reform primary and secondary education. Most significantly, this legislation will create a new system for statewide accountability and assessment that will, for the first time, measure individual student progress over an extended period of time. That is critically important,” Beshear said in a statement.

Politically, some wonder anew if House Speaker Greg Stumbo's rookies should be faulted for the surprise abandonment of accountability over the next three years. Was this a phantom provision, dropped into the free conference report at the last minute? If so, how, and by whom? Or was it a provision that all conferees agreed to with eyes wide open?

"It's an outstanding bill that will make great improvements and will honor the input that we've received from our educators," Rep. Harry Moberly, D-Richmond [told the Herald-Leader].

Republican Sen. Dan Kelly of Springfield, a longtime proponent of changing Kentucky's assessment tools, said the bill would strengthen Kentucky's education system, not weaken it. "This is not the death of reform," Kelly said.

By all accounts, there was a great deal of input from teachers who had had enough....of something. Some clearly reacted to the amount of time they were being "asked" to commit to assessment at the expense of instruction. After all, you don't fatten a calf by weighing it. Some claimed that district personnel and principals had pushed teachers to the edge of some ethical cliff, and they pushed back.

Kelly's assurance that this does not mark the death of education reform in Kentucky is a welcome acknowledgement for those of us who believe a strong system of public schools is essential to our ability to compete as a state. But not all conservatives are buying it. Over at vere loqui, Martin Cothran declared that with the removal of the testing system KERA is officially dead and asked,

Will the last person out of Kentucky's Education Reform Headquarters please turn off the lights?

At KentuckyProgress, David Adams noted "the end of a disastrous episode in education 'reform'" and "got a kick out of "the Courier Journal referring to Bluegrass Institute as merely a "right-wing think-tank" (when everybody knows they are so much more). He thanked the Herald Leader for the "big laugh and free mention" when they referred to "conservative enemies" which Adams correctly assumed meant BIPPS, even though H-L didn't say.

None of the conservative groups came right out and said that Kelly was wrong. But they apparently believe he was.

The Herald-Leader lamented that as "lawmakers were dismantling Kentucky's school-accountability system last week, researchers were holding it up as a national model" citing Kentucky as one of five states that stand out for increasing high-school graduation rates while reducing schools that are "dropout factories." "It's impossible to not credit the Kentucky Education Reform Act for improvements in graduation rates," the H-L wrote.

Secretary of State Trey Grayson issued a statement voicing his dismay over the reduction of emphasis on social studies.

“By suspending the state accountability index and following the federal accountability requirements, we run the risk of marginalizing civics education,” he said in the statement. “With no accountability system in place for social studies, schools will naturally place a greater focus on those subjects that are part of accountability standards of No Child Left Behind.”

It now falls to the Kentucky Board of Education to guide KDE's response to Senate Bill 1. They start chatting on April 1st.

In the interim, some school districts, like Fayette County, are going to "make public its own 'academic index' based on this spring's testing, whether the state publishes a performance index or not." State education department spokeswoman Lisa Gross told the Herald-Leader the interim provisions in SB 1 might lead to widespread confusion until the new assessment system starts. Wayne Young, executive director of the Kentucky Association of School Administrators told H-L, "Some districts are going to go ahead and finish their portfolios. But you could get a bit of everything. There's no pattern that I would describe."

This morning outgoing President of the Council for Better Education Roger Marcum told state education leaders in an email message that Kentucky "chose the path of least resistance and the high risk of lower expectations for the immediate future."

His opinion echoed that of Prichard Committee head Bob Sexton who wrote an open letter posted on the Prichard blog,

It was decided that, for the next three years starting this spring, Kentucky schools should have no accountability except for student test scores on math and reading as required by No Child Left Behind. This means that the results of testing for writing (portfolios included) science, history, geography, economics, civics (i.e., social studies) and the arts won't be counted. The Kentucky Department of Education is also prohibited from publishing an overall school improvement score (accountability index) so schools won't know whether they are better or worse than the previous year - and neither will parents or other taxpayers.

Sexton quoted KEA President Sharron Oxendine as saying, "we should 'just give everybody a breather' from most accountability until we have a new system in three years."

The problems with that, according to Sexton include:

1. For the next three years schools won't have a way of telling parents and other taxpayers how they're progressing with student learning except in math and reading required by NCLB.
2. A likely result will be to de-emphasize writing, science, social studies (history, economics, geography, civics) and the arts.
3. Strong accountability ... and making the case for adequate or superior school funding are joined at the hip.

This last point has some serious legal implications for schools. Kentucky's constitution requires the General Assembly to provide an efficient system of common schools throughout the commonwealth. The Supreme Court has ruled that that means schools must be adequately funded to reach their goals. The absence of state accountability makes it very difficult, if not impossible, for a court to determine whether the General Assembly has fulfilled its obligation. How is a court to determine whether the system is sufficiently funded to reach its goals if the state has no justiciable standards by which the court might measure the efficiency of the system?

Who Killed the CATS?

It's a shame the CATS assessment went out as it did. Better would have been a planned move to segue CATS toward a kind of value-added system that collected lots of data, but which restrained any inappropriate, scientifically unsupportable uses of the data. C'est dommage.

The state's major newspapers provided their own post mortems on school reform in Kentucky since Friday's collapse of the CATS assessment. They found the same perpetrators.

Their analysis is correct, to a point, but incomplete.

In "Death of reform" the Courier-Journal offered,

The irony is, it's the Republican leadership in Frankfort that has linked arms with the statewide teacher union, the Kentucky Education Association, to trample the remaining vestiges of what was once a comprehensive state reform, with a demanding accountability system.

It's understandable why the conservative, educationally revanchist Republicans and their right-wing think-tank friends want to discredit Kentucky's public education reform, as part of a long-term strategy to abase the public sector, mollycoddle middle-class striver-parents with outmoded, ineffective nationally normed testing and pave the way toward private school vouchers.

C-J might also have mentioned that some opponents didn't seem to believe that equality of educational opportunity was a particularly useful goal from the start. As early as 1994, the Bluegrass Institute's Richard Innes was identified by the Kentucky Post as a KERA foe. BIPPS has since shown a willingness to twist or exaggerate the truth out of any data point to tell a story they wish to be true. The Family Foundation proudly counts itself among those who opposed KERA from the start.

The Herald-Leader has learned that good teachers are sad, but bad teachers are happy. They didn't specifically indicate which were which.

With no whisper of objection from Gov. Steve Beshear, lawmakers last week kicked the ladder out from under the many educators who have been working hard to pull their students and schools up. (On the lower rungs, less accomplished teachers were gleeful.) ...

As it stands, cynics can justifiably conclude that Beshear and House Democrats, under new Speaker Greg Stumbo, caved to the worst impulses of both the teachers union and conservative enemies of public schools.

Similarly, C-J declared it "a mystery why KEA would want to help them do their dirty work," but then decided it must be teacher laziness, and resistance to new ideas - not from the good teachers, you know, but the bad teachers.

You can tell who's who by their support for CATS - or lack thereof.

Simple.

And too simple is what makes both papers wrong.

The papers' desire to turn a deaf ear to the increasingly shrill complaints from a majority of Kentucky teachers - good and bad - over the most recent escalation of test preparation tactics has served to support overzealous superintendents and principals as they ignored veteran teachers' opinions while applying the thumb screws to any teacher who dared not comply.

In that process, we have seen school districts introduce new program after new program, rarely allowing faculty time necessary to allow the innovation to become intergrated into the system. They didn't have time to wait. The superintendent wanted another 10-point jump this year. Somebody on the editorial boards should have written about that.

The holy grail of school reform became that too-good-to-be-true savior principal whose leadership alone would lift children out of ignorance and set the world on the right course.

It a great dream. But it's a dream.

In that sense, do the papers share some complicity for the demise of the CATS assessment?

Just as the papers' unbending support for the Primary Program drove away far too many of our school's most energetic supporters - the parents of primary children - the gamesmanship of the CATS assessment soured a majority of Kentucky teachers. One wonders where we would be today if a more balanced approach was employed by our school reformers, our administrators and the editorial boards. Might it be that the conservative forces could not have mustered the necessary support for a unamious vote in both houses without the number of disillusioned teachers the system has created?

Did you hear that big sigh of relief sweep across the state? I did.

Is it party time in Kentucky schools? I hope not, but I wouldn't give a plug nickle for program evaluations as a substitute for real accountability.

Agreement Reached on Scraping CATS

“It’s a great bill. It’s exactly what we wanted.”

---Rep. Carl Rollins, D-Midway

That according to House Education Chair Carl Rollins as agreement was reached this afternoon on on a bill to replace the CATS accountability testing system.

This from Tom Loftus at C-J:

...Under the compromise, students will take the current test this spring in the areas of math, science, reading, social studies and on-demand writing.

Districts can choose to test in the areas of practical living and career studies and arts and humanities.The exam won’t be used for state assessment purposes, and writing portfolios will be used for instruction only.

The exam will last for seven days instead of the current 10.

In the 2009-10 school year, schools will use the current test in math, science, reading, social studies and on-demand writing.

A national test will be given for the purpose of providing parents and students a measure of individual student performance.

The exam in 2009-10 will last six days.

In 2010-11, schools will use the current test in science, reading, social studies and on-demand writing and will give a national test.

In addition, a math exam based on new standards, to be developed by the Department of Education, will be given as a sort of test run. The testing schedule will drop to five days.

Sexton Decries Early Death of Accountability

The Prichard Committee has been generally supportive (with reservations) of efforts in the House to produce an alternative to the Senate Bill 1. Today they are wondering what hit them.

Somewhere, Sharron Oxendine is smiling.

This from Bob Sexton:

CATS: What today's news stories missed?

In a surprise last minute move on legislation affecting testing and accountability (incorporated now in SB 1), the House amended the bill to "suspend" accountability in most subjects for at least 3 years starting this spring. The consequences will be highly disappointing for parents, students, and the public because these measures of school progress underpin Kentucky's push for improved student learning --- and support the case for adequate school funding.

But now it will be more difficult for parents, taxpayers and teachers to figure out how their schools are progressing in core subjects like writing, science, history, geography economics and civics, and there will be no measure of how they are doing overall (calculated by the index of all the subjects).

All they will know under "suspended" accountability is what No Child Left Behind testing in math and reading tells them, which isn’t enough. There’s a strong chance too that all the other subjects---the ones that "don't count"--- will get less attention in the classroom.

This is a hugely ironic outcome for the teachers who lobbied for this change. Teachers who have bitterly complained about NCLB now get nothing but NCLB! They also saw their best case for adequate funding --- evidence of making progress with children --- thrown in the garbage. Be careful what you wish for, as the old saying goes.

So with data on school results "suspended" what's next? Who knows. What information will school boards and superintendents use to push local schools to improve? Will we suspend aid for struggling schools and efforts to reduce achievement gaps? Will we suspend calls for adequate school funding?

Stay tuned --- schools and kids are in for a bumpy ride.

SOURCE: Prichard release

Ding Dong...

This from Ronie Ellis in the News and Tribune:

House passes bill to replace CATS test

Differences with Senate version

must be ironed out before final passage

FRANKFORT — The widely detested CATS school accountability test isn’t officially dead just yet – but it’s close.

The state House on Wednesday approved legislation to narrow the focus of the test, allow teachers to concentrate on fewer standards and students to gain a deeper understanding of those standards, according to Rep. Harry Moberly, D-Richmond. The test will cost less, take less time, and match up with Kentucky college and university expectations for incoming freshmen.

It removes writing portfolios from the accountability index, although they will continue as an instructional tool. Some open response questions will be retained, although fewer than in the CATS test. And it must be administered during the last two weeks of the school year.

The bill is largely the product of negotiations between the House and Senate over differences between the two chambers’ approaches to school testing. On Wednesday the House education committee inserted into Senate Bill 1 language reflecting those negotiations, although committee Chairman Carl Rollins, D-Midway, said there are a few differences still to be worked out.“

Basically, this is what we proposed,” said Sharron Oxendine, president of the Kentucky Education Association. “You can hear teachers cheering all over the state. They know it’s going to be a better day for instruction.” ...

Grayson, Prichard Committee say bill could hurt science, social studies

This from Bluegrass Politics:

Bob Sexton, executive director for the Prichard Committee for Academic Excellence, said the legislation aimed at replacing the Commonwealth Accountability Testing System (CATS) could have an unintended consequence of prompting Kentucky schools to focus more on math and reading to the detriment of the other subjects over the next three years before the new testing system is developed for the 2011-2012 academic year.

The House passed a re-written version of Senate Bill 1 by a 97-0 vote Wednesday.

The bill scraps the CATS test and immediately jettisons the writing portfolio requirement from the testing scheme. Other subjects, such as arts and humanities and living studies, also will be taken out of the testing formula next year, according to the bill....

...Secretary of State Trey Grayson weighed in with a similar appeal Wednesday, as reported by Bluegrass Politics.

“By suspending the state accountability index and following the federal accountability requirements, we run the risk of marginalizing civics education,” Grayson said in a statement. “With no accountability system in place for social studies, schools will naturally place a greater focus on those subjects that are part of accountability standards of No Child Left Behind.” ...

Amended HB 508 Clears House Education Committee

This from Jim Warren at the Herald-Leader:

House Committee adopts student testing bill

FRANKFORT — A bill that would replace Kentucky's student testing system but retain some of its key principles sailed out of the House Education Committee on a unanimous vote late Tuesday.

House Bill 508 not only would create a new testing system, starting in the 2011-2012 school year, but calls for creation of new curriculum standards that would be used for the test...

...[The bill's lead sponsor, Rep. Harry Moberly, D-Richmond] said after Tuesday's vote that the House Education Committee probably will place the "substance" of HB 508 in the Senate's testing bill as a committee substitute later this week.

The Senate measure, Senate Bill 1, also would abolish CATS. But it seeks to replace open-response questions with more multiple-choice exams that its backers say would save time and money. SB 1 already has passed the Senate.

If an amended version of SB 1 clears the House, a committee of lawmakers from both chambers would probably negotiate a compromise bill later this month...

...the House committee approved several technical changes to HB 508, plus a longer amendment offered by state Rep. Jody Richards, D-Bowling Green, aimed at strengthening writing.

HB 508 would retain writing portfolios for instruction purposes but take them out of the state accountability assessment program. Richards said he feared that could lead to writing being de-emphasized. His amendment calls for various changes to help improve how writing is taught...

Former Commish Finalist Likes SB 1

One of the finalists for Kentucky Education Commissioner, the last time around, commented in today's Messenger-Inquirer about proposed legislation to change the CATS assessment.

Owensboro Superintendent Larry Vick likes a lot of the provisions in Senate Bill 1, including use of a norm-referenced test.

"A lot of people say with a multiple choice test you can't have higher-level thinking, but some well-designed tests are fairly complicated and involve concepts," Vick said. "We need a balanced assessment."

Vick ...[is] hoping lawmakers do more than just focus on CATS...

Vick said Senate Bill 1 is moving in the right direction and is an improved bill from last year. The easy part, though, is redesigning the test, he said.

"The harder part, which no one has talked about yet, is the assessment system," Vick said.

"How do we hold people accountable? How do we assess the results?"I think the Senate has made a real compromise based on last year's discussion. Now we should take a couple of years and thoughtfully develop a new approach to assessment."

It is unclear why Vick thinks no one is talking about the assessment system. It is also too soon to know if Vick will be in the commissioner mix this time around.

C-J Chooses the Lesser of Two Evils

This from the Courier-Journal:

Better, not good

If the General Assembly is determined to steamroll ahead with a major change in education policy, even though there's too little time left to do the job right, House Bill 508 is the better vehicle to use.

A comparison and contrast by a Prichard Committee expert suggests ... the bill pushed by Rep. Harry Moberly, D-Richmond, is preferable to Senate Bill 1, which is covered with the handprints of those who long have opposed the historic Kentucky Education Reform Act of 1990...

...The Moberly bill tries to ensure that higher education experts are involved in setting standards to promote college readiness, and that teacher training programs are producing graduates who can meet high standards. But there's no real assurance these things will materialize.

There's also no assurance that funding will be available to turn all the rosy rhetoric into real educational progress.

Most important, there's no way to be sure this bill avoids the withering erosion to which KERA has been subjected over the years.

But it's by far the better of the two alternatives at hand.

C-J points to HB 508's

• systematic and inclusive approach to the content
• that it allows enough time to revise current standards
• mandates the use of testing that gets at problem-solving and communication skills
• emphasizes so-called "criterion-referenced" testing
• offers some attention to writing portfolios
• promises a "review" of programs in arts and humanities, and in practical living and vocational studies
• Opens the way to adopting end-of-course exams

C-J finds the removal of writing portfolios from the accountability system in both HB 508 and SB 1 tragic, "most likely eliminating any effective classroom emphasis on them," and they seem to share my skepticism of the program review.

They also object to "using the old-fashioned and unsophisticated multiple-choice-only approach" utilizing "norm-referenced testing," which narrows accountability and "places the student in relationship to other test-takers."

The New Commonwealth Assessment

By Skip Kifer

Lord Mansfield is reputed to have said “Decide promptly, but never give any reasons. Your decisions may be right, but your reasons are sure to be wrong.” I hope his admonition is correct because I get a sinking feeling each time I see another list of what form the Commonwealth’s new assessment should take.

The reasons for the sinking feeling are many. Here I will deal with just three.

The first has to do with the technical aspects of whatever form the new assessment takes. I believe the substantive ideas behind the assessment, not its technical requirements, should drive it. Yet, good technical advice early can save time, money and embarrassment. In the past the Commonwealth has been blessed with unusually good technical advice from panels that were composed of assessment persons who were in the mainstream of the assessment world. They gave dispassionate advice that vendors or persons not in the mainstream could not be expected to give. There should be technical assistance early and often as plans for a new system develop.

There are a number of proposals that make me wonder about the basis for them. To take just one: that formative assessment instruments could or should be provided by vendors. I have taught test and measurement classes for over 30 years and I know that I can teach teachers to make better tests than are presently being used (apart from CATS) in the Commonwealth.

Some very important things go on when one constructs a test. One needs to think seriously about standards, instructional materials, and the learner as the items are written and tests developed. Having gotten the results one has to think seriously about whether students have not learned because they have not been taught well or because the items are no good. Each of these activities is related to effective instruction. None of them come with already made assessments.

I would like to hear the discussion of whether difficulties with the writing portfolio are unique to that form of assessment or whether having assessments in any accountability context corrupts them regardless of their form. Are some kinds of assessments less corruptible than others? Are there practices that are less likely to be gamed by those who seek higher scores at the expense of better learning? Are there ways to gather and report learning outcomes that depend less on massive formal testing procedures? I believe the answer to each of these questions is “yes.” And I think one could give good reasons for answering these questions in the affirmative.

And so, I will enter the fray. Below I am presumptuous enough to come up with a set of principles that should guide the design of the new Commonwealth assessment. I made the mistake of including things we have learned as my reasons for the recommendations. I wonder if there is a Lord Mansfield redux.

New Assessment Principles

After almost two decades of high-stakes assessment in the Commonwealth it is time to step back and decide how what we learned can help us think about the new directions we should take.

1) We learned that assessments can be very time consuming and very costly. We learned that they may not be as cost effective as we hoped.

A new statewide assessment should be:
a. Clear about each of its purposes
b. Less time consuming
c. Less costly

2) We learned that teachers believe they spend far too much time testing and preparing for testing.

A new statewide assessment should
a. Carefully delineate what should be assessed statewide and what should be assessed locally and be controlled by teachers
b. Be restricted to a small but crucial part of the curriculum

3) We learned that parents wish to know how their children’s scores compare with other children’s scores nationwide. We know that off-the-shelf tests have user norms not nationally representative ones. We know that the National Assessment of Educational Progress (NAEP) is the only national test with a nationally representative sample.

A new statewide assessment should
a. Use wisely the NAEP state assessments
b. Link Kentucky assessments to NAEP
c. Provide students with scores that can be compared to national ones

4) We learned that measuring so many content areas at so many grades in all schools is an inefficient way to assess Kentucky’s educational progress.

A new statewide assessment should:
a. Sample schools to track progress over time
b. Accept NAEP scores in mathematics, reading and science as progress indicators

5) We learned that testing requirements of No Child Left Behind (NCLB) have a profound effect on what and how a state assesses.

A new statewide assessment should be
a. Cognizant of new requirements of NCLB
b. Prepared to seek exemptions from some requirements of NCLB

6) We learned that others admired Kentucky’s assessment when it was bold and pioneering

A new statewide assessment should

a. Lead other states and nations in producing useable information about students and schools
b. Emphasize formative and instructionally embedded assessments over summative ones
c. Place students and teachers in the center of schools and assessment

Jack Foster's Idea: "a cadre of specialists"

This from Jack Foster in the Courier-Journal:

...Rather than continue to search for a cheaper and less time-consuming state testing system that is not helpful to students, parents or teachers, I believe it is time to find a way to improve the reliability and validity of classroom assessments so they can be used for accountability purposes and eliminate the need for a separate testing system for state accountability purposes. How might this be done?

I propose that classroom assessment responsibilities be given to a cadre of specialists who would administer state-designed assessment instruments at regular intervals throughout the year in every classroom in the commonwealth. The content of these tests would be tightly integrated with classroom instruction and be directly aligned with state core content standards for each subject and grade level.

Teachers would continue to assess student progress on a daily basis, but report card grades would be based on the results of these tests, thus motivating students to do their best on them.

Finally, "grading" of student classroom learning would be uniform and comparable among all classes, schools and districts across the commonwealth. Data generated by these tests could be aggregated and analyzed for school, district and state accountability purposes and to improve the assessment instruments, while eliminating the need for a separate, once-a-year state test.

This approach has many potential benefits. It could relieve teachers of the time-consuming task of creating, administering and scoring their own classroom assessments for which they have insufficient time and expertise; provide greater consistency across the state in what is tested and how it is tested in each subject at each grade level; provide almost immediate feedback to students and teachers to improve learning and instruction; bring a high level of rigor and validity to classroom assessments; reinforce a focus on teaching core content; and the tests would matter to students.

Money currently spent on CATS could be used to develop the classroom assessment instruments and money currently spent on teaching assessment skills to teachers could be used to support a cadre of assessment specialists throughout the state. Much of the testing could be done electronically.

The transition from a CATS-like accountability system to one envisioned here must be carefully planned and should be developed in parallel with a revision of the core content standards that has been proposed.

Doing it right is more important than doing it quickly...

CJ Smacks ...Everyone over Assessment Reform

This from C-J:

...Here are some facts:

It does not take a good teacher excessive time to prepare for CATS tests, as critics like to claim.

It would be wrong to drastically narrow the testing, since that inevitably would narrow the teaching that precedes it.

It would be astonishingly stupid to remove writing portfolios from the accountability system, because that certainly would end serious classroom work on this profoundly important capstone skill, which is essential not just for admission to college these days but also to success in the information age workforce.

It would be equally absurd to remove open-response test questions, since they get at the very deep learning and critical thinking that education reform was supposed to foster.

It's deceitful to condemn the legitimate coaching of student writers as "suspect" and to mistake the product of student rewrites as "inauthentic."

It would be educational malfesance to replace CATS with a jury-rigged combination of some off-the-shelf nationally normed test and a few extra questions that make passing efforts to address Kentucky curriculum -- especially since those nationally normed tests are designed to categorize a substantial percentage of those who take them as failures.

Throwing open the entire state curriculum to a revision of standards would, in the current environment, certainly produce a dumbing down of what Kentucky expects its kids to learn...

... there are deep, important differences separating what educators, politicians and competing advocates believe should be done.

To run a political bulldozer over CATS in the few days remaining to this General Assembly session would be a travesty.

Incredibly, almost nobody is standing up to object. All the interested parties cower at the roadside, frightened of retribution from Mr. Williams, Mr. Kelly and the regressive caucus that keeps them in control of events at the Capitol.

Conservative enemies of public education sit in the cramped turrets of their think tanks and cheer.

Perhaps the most pathetic claim is Mr. Moberly's -- that his House bill will successfully set and enforce "world-class benchmarks" and "preserve the principles of reform," serving not only to place Kentucky students' achievement in a national context but also to supply prescriptive information for individual students and to test the acquisition of state-established curriculum. ...No testing system can claim to do all three...

Perhaps the greatest disappointment is a lack of leadership from Gov. Steve Beshear, who has contented himself with a muted call for a thorough review ... whining, "The Republicans won't let me do anything." ...