Build a Value-Added Assessment System for P-12 Before Moving on to Teacher Prep Institutions

"A key assumption of using test scores to judge teachers

is that students are randomly assigned,

first, to schools, and, second, to classes.

Neither is true."

Recently, there has been increased talk of holding teacher preparation institutions accountable for the performance of Kentucky teachers - in effect holding colleges accountable for student achievement in the state's public schools.

It's a good motivation, but in practice, there's a lot wrong with the idea.

It simply takes all of the problems associated with teacher-accountability-by-standardized-test-score, a central tenant of the Obama Administration, and multiplies that unfairness many times over. While it may feel good to teachers to know that they are not the only ones tied to an unfair system, it does not fix what's broken.

But it may become a better idea, if those who are enthusiastic for change slow down long enough to create a system that addresses fairness by trying to address the many technical problems, and that might actually work to an acceptable degree. That means building Kentucky's new accountability system from the bottom up:

• Curriculum standards, first
• Assessments built on those standards
• Lessons taught on those standards (in that order)
• Assessment results fed into a value-added accountability system that is sensitive to the great variability in children and one that establishes an individual baseline for each child and controls (to the degree possible) for that demographic variability.
• At this point the public should consider a cost benefit ratio: the investment, in relation to the reliability and validity of the system.
• Professionals should gauge the practical limitations of social science research, both quantitative and qualitative. Unlike the natural sciences, our variables refuse to hold still; which bears heavily on the precision of the system.
• Individual student progress is measured from each individual student's established baseline
• Individual student achievement data is collected over the student's entire academic career
• The teacher accountability system should be quantitative and qualitative.
• The principal accountability system should be quantitative and qualitative
• There should be a planned review of the accountability system after collecting about three years of data (barring some unforeseen data catastrophe), with any significant adjustments to the accountability formula made at that time.
• Then, and only then, a teacher preparation institute accountability system should be ready to track the performance of teachers who graduated from each institute, based on that value-added system; and that system should be quantitative and qualitative

Tying teacher accountability to test scores is going to happen. Fairly or unfairly, states are now scrambling to meet the strictures of RTTT grant funding. But desire for much needed funding doesn't make the myriad technical problems go away.

The Century Foundation recently outlined their Eight Reasons Not to Tie Teacher Pay to Standardized Test Results. In a nutshell,

Reason #1: Tying test scores to teacher compensation suggests that teachers are holding back on using their experience, expertise, and time because they are not being paid for the extra effort.

Reason # 2: The standardized tests in most states are lousy and so are the standards they are designed to measure.

Reason #3: The idea of compensating teachers individually in order to differentiate their performance from their school colleagues defeats a principal tenet of good instruction—that teachers need to learn from one another to solve difficult pedagogical challenges.

Reason #4: Most teachers do not teach a grade or subject that is subject to standardized testing.

Reason # 5: Even reliable standardized tests are valid only when they are used for their intended purposes.

Reason #6: A key assumption of using test scores to judge teachers is that students are randomly assigned, first, to schools, and, second, to classes. Neither is true.

Reason #7: State data systems are in their infancy. It turns out that it is harder, is more expensive, and takes longer for states to produce reliable, accurate, and secure longitudinal data on students and teachers than widely assumed.

Reason #8: The rationale for tying tests to compensation is not clear.

The non-profit, non-partisan Century Foundation argues that No Child Left Behind has narrowed instruction too much already, that one does not need a standardized test to identify the worst and best teachers, and no system could be constructed with sufficient precision to withstand the inevitable court challenges.

At the heart of the argument in favor of tying pay to test scores is the idea that it will improve practice. But that can only work if the economy provides the anticipated financial incentives. In this recession,

"if teacher compensation does not keep up with inflation because of poor student performance, then teachers will . . . what? Work harder? Dig deeper? Stay longer? There is no evidence that such measures improve instructional practices or student outcomes."

Secretary Duncan is correct when he catalogues the weaknesses in the present system of preparing, recruiting, mentoring, retaining, inspiring, retraining, promoting, and dismissing teachers. but this is an idea that is way ahead of just about everything it would need to have even a chance of working fairly and reliably, if at all.