Sunday, August 05, 2007

What Testing Guru Bill Sanders Really Meant About Multiple Measures

This from Alexander Russo at This Week in Education.

Once in a while, I actually do some reporting, and today I happened to talk Prof. William Sanders, the testing guru whose recent letter to Congressman Miller was leaked to the press and seemed (according to an Ed Daily story) to put Sanders squarely against Miller's proposed use of multiple measures in AYP.

Well, it turns out that Sanders is against the use of portfolios and classroom observations that are often called multiple measures, but not against end of course tests, college entrance tests, and the like that he thinks Miller is talking about. "Those things have a place," says Sanders, who points out that they are already part of the growth model projections that he has developed and are being used in some pilot states.

To those who are concerned about the complexity and transparency of both the current AYP and proposed changes, Sanders says such intricacies are the price of a nuanced and reliable rating system. "A simple system could be developed," he says, noting that some states are going that direction, "but it would be less reliable and more biased [than a more complex one]."

His main accountability concern, however, is not so much that the current AYP relies on "a single test" (a description he says irks him and ignores the fact that there are three years of tests and hundreds if not thousands of test item responses that go into each year's AYP calculations), but rather that it encourages too much focus on lower-performing kids rather than "early high-achieving kids" who get ignored. He proposes a rating system that evaluates schools not only on reducing the achievement gap but also on helping already-proficient kids do even better -- [See] ... Tennessee ...and Nevada.

This from the Center for Greater Philadelphia at the University of Pennsylvania:


Value-Added Assessment in Tennessee


Tennessee is the state most strongly identified with value-added assessment. Its system dates back to 1992, when value-added was implemented as an integral part of a comprehensive education reform measure. Using a complex statistical method developed by Dr. William Sanders, then a statistician at the University of Tennessee , the Tennessee Value-Added Assessment System (TVAAS) provides:

Data to the public on the performance of districts and schools, and data for appropriate administrators on the performance of teachers;

Information to teachers, parents and the public on how schools are doing in helping each child make academic gains each year;

Information to school administrators to help identify weaknesses in even the strongest schools.

TVAAS is a statistical methodology that begins with testing each student in each grade in a number of subjects. Through 1997, Tennessee tested second through eighth grades in Reading , Math, Language, Science, and Social Studies. Tennessee began testing grades three through eight in 1998.

The TVAAS statistical model aggregates student growth increases using a design that accommodates missing data. Because of a philosophical belief that schools should insure that all students progress at equivalent rates, no matter their disadvantages, the model does not include other data on students.

In a report by the Council of Chief State School Officers, Tennessee's 8% increase in math and science scores was linked to TVAAS. In addition, Tennessee is one of the few states that have shown improvement on the National Assessment of Education Progress since TVAAS was implemented in 1992.

The state has both rewards, aid, and sanctions linked to its school rating system.There is no specific value-added teacher evaluation as part of Tennessee 's accountability system, but school administrators have access to teacher level data that can be used to improve instruction. Value-added scores can be used for up to 8 percent of a teacher's evaluation.

The incentive funds are only available to schools, not to teachers.

Although teachers and administrators were suspicious at first, they are now finding they can actually use TVAAS to improve teaching, something that no other accountability system has afforded.

No comments: