Tuesday, January 24, 2012

Can a Few Years’ Data Reveal Bad Teachers?

This from the New York Times:

With years of data, it seems possible to distinguish good teachers from poor ones. Does that indicate that, after collecting two or three years’ data on each new hire, districts should be using test scores for decisions about firings, tenure and pay?

The following in an online "debate."

The Value of Test Scores

Raj Chetty Raj Chetty and John N. Friedman, economists at Harvard, and Jonah E. Rockoff of Columbia co-wrote the recent study "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood."
John N. Friedman
We all agree that teachers can make a tremendous difference in the lives of students, and we all can remember a great teacher who was important in our own lives. The challenge is to identify more great teachers. Value-added measures, which rate teachers based on their impacts on students’ test scores, can help us do so. Our recent study shows that when a high-value-added teacher enters a school, test scores for students in the grade taught by that teacher rise immediately (as shown in the figure below). And the gains don’t stop there: the students who learn from that teacher are more likely to attend college, earn more, and are less likely to have children as teenagers. Even when new teachers are evaluated with just a few years of data, those who get high value-added ratings produce large gains for their students...

Let’s Not Rush Into Value-Added Evaluations

Jesse RothsteinJesse Rothstein is an associate professor of public policy and economics at the University of California, Berkeley. He has studied the relationship between classroom assignments and estimates of "value-added" by teachers.

The new Harvard-Columbia study provides important information about the relationship between student test scores and longer-run outcomes. But there is much that we still don’t understand. We need careful study of pilot programs, not to remake our education system.
One unheralded new result is that teachers’ effectiveness changes over time. The study shows that some teachers are ineffective at first but improve as they age, while others start strong but then burn out. Policy design must account for this. How many teachers who are fired early on for poor student achievement would improve given the chance? And how many who get early raises will continue to draw them through years of later coasting? Calculations that firing a poor teacher saves $2.5 million entirely ignore this factor...

Dawn ShirkDawn Shirk teaches English as a second language at Reidsville Middle School and Reidsville High School in Reidsville, N.C.

Traditionally, teachers have been observed by their principal once a year, and evaluated solely on that one encounter. Long-time teachers would often go years without having an observation, or even a casual walk-through by an administrator. Fortunately, times are changing.
In North Carolina, we are in our second year of using a new teacher evaluation system, developed by Mid-continent Research for Education and Leadership. Through a series of rubrics, teachers are evaluated in five areas — leadership, respect for diversity, understanding the content they teach, the lessons they create and execute, and their reflection on their own teaching. Sub-topics include use of technology, global awareness, relationships with parents, and the use of data from standardized tests to drive instruction. It is a yearlong process that includes several observations, by principals and peers, conferences between administrators and teachers, and a collection of artifacts that teachers present to demonstrate the many things they do that may not be seen in an observation, which includes parent conferences, teacher collaboration, working with community organizations, professional development, committee work and much more...

We Know Which Teachers to Fire

Lance T. Izumi is the senior director of education studies at the Pacific Research Institute.

Lance T. IzumiA study of 2.5 million students by Harvard and Columbia researchers strongly indicates that individual teacher quality based on student test scores significantly impacts students’ life outcomes. While that should be enough evidence for school districts to consider using testing data to help inform firing, tenure and pay policies, the key will be overcoming teacher union opposition.
Analyzing two decades of test results, the researchers found that improving teacher impact on test scores, referred to as “value added,” even in a single grade raises the probability of students attending college and increases their future earnings. The data also suggests that getting rid of the lowest-performing teachers would significantly raise students’ lifetime incomes. Yet, many teacher unions are likely to be unmoved by these findings...

Results Are In; How Will We Respond?

Arun Ramanathan is the executive director of The Education Trust—West.
Arun Ramanathan

There is plenty of strong evidence that we can use data to assess the impact of teachers on student outcomes. In a recent report, The Education Trust—West found that on average, students placed with the strongest teachers gained half a year more in English than students placed with the least effective teachers.
Differences of this magnitude are important for all students, even our highest achievers. But they can be devastating for low-income students and students of color who often enter school already behind. With a series of effective teachers, struggling students can quickly catch up to grade level. But one or two weak teachers can prevent them from reaching their academic potential.

Unfortunately, our research revealed that African-American, Latino and low-income students are far less likely to have access to the best teachers. Just as worrisome, high-need students can lose access to highly effective teachers as the result of quality-blind layoffs based solely on seniority. No wonder we have failed to close the achievement gap...

Use the Data, but Constructively

Sydney MorrisSydney Morris is a former public school teacher and the co-founder of Educators 4 Excellence, a national nonprofit that seeks to elevate the voices of teachers in education policy.

The recent study by Harvard and Columbia economists showed a link between quality teaching and higher test scores and between higher test scores and positive life outcomes. Researchers found that students with top teachers are less likely to become pregnant as teenagers, more likely to enroll in college, and more likely to earn more money as adults. These new findings highlight something that we as teachers have always intuitively known – that what we do everyday in our classrooms has far-reaching impact.  
Despite this knowledge, a false dichotomy exists between proponents and opponents of using student-growth data to evaluate teachers. We often hear of the “reformers” who want to use student test scores to identify and fire the lowest-performing teachers, and conversely, the teachers’ unions who are painted as defenders of the status quo.  
Lost in this back and forth are the voices of real classroom teachers who want meaningful evaluations that give them the feedback and support they need to improve their craft. In Educators 4 Excellence’s work with nearly 4,000 teachers in Los Angeles and New York City – the nation’s two largest school districts – we have consistently heard from educators that they believe their students’ academic growth should be one, among many, indicators of their performance...


Skip Kifer said...

I spent a little bit of time looking at the economists study. The picture they refer to shows a range of scores (I assume they are residuals) from .19 to .23. So what might the .04 be?

Of course the major flaw of the study is to make causal inferences from correlational data. It can be done, but should be done humbly.

I am stunned that they suggest they can predict so many years later. That don't do that well with the economy. Why should they believe they can do better with human beings?

I do believe teachers come to their task with varied interests, attitudes, and previous models of instruction. I also think there are lots of ways to be a good teacher and few of them are related to higher test scores.

Anonymous said...

This all seems rather amusing how there exists a perception that student test scores somehow indicate teacher effectiveness or that this type of professional competition exists in the rest of the employment world.

According to this approach, we should fire police officers who serve in areas with higher than average crime rates. As mentioned in an earlier posting, we will need to take away the medical licenses of physicians who care for patients who continue to be ill or simply die. Certainly we need to terminate all those psychiatrists, psychologists and counselors because it seems like all their patients do is come back to see them with the same problems. What about all our university colleagues who haved fail students who performed poorly in their courses, failed to graduate or did not achieve high enough scores on professional licensure/certification exams. Don't even get me started on legislators - that would be like shooting fish in a barrell.

By the way, where are all these great unemployed teachers we are going to be using to fill the open slots? Or could it be that many of the great teachers left the profession because we created conditions which limited their ability to be great?

Richard Day said...

Yeah. And even if their model worked the advantage would come to about 250 bucks a kid. Hardly headline grabbing - so they multiply the number of kids until the number sounds better.

The one thing I thought I saw was yet another confirmation that nothing is more important than teacher quality. That's where our focus ought to be.

Mary said...

How would you measure that teacher quality. The current evaluation system is flawed and extremely subjective.

Anonymous said...

Excellent contribution in today's (1/30/12) Lexington Herald from retired UK professor identifying the obvious flaws in teacher evaluation which uses student standardized performance or annual growth rate.

Anonymous said...

Hate to think that my first year teaching combined with that particular year's student grouping I was dealth would have determined if I would continue as a teacher.

My efforts may not have resulted in a year's worth of growth but their performance was equally not reflective of my effort as it was their own ability and the efforts of those who taught them before me.

If they want to evaluate me like an hourly worker in a factory, then that is the type of mentality and effort I am going to give them. Not going to serve on commitees or do extra curriculars, not going to worry about parent commnication or staying after my required contractual time. Not going to assign any assignments I can't grade beyond my one hour of planning and definatly not going to take on any student teachers who might screw up my programing of kids for the exam or jepardize my continued employment.

Ally Page said...

By grading our teachers based on their student’s test scores we are potentially dismissing high quality educators. Test scores do not reflect everything a student knows and can be influenced by outside factors unrelated to the teacher. This method of evaluation frightens teachers into “teaching to the test” which could cause them to leave out other important areas of learning such as creativity, problem solving, and critical thinking skills. Teachers should be evaluated in many different areas including how they form relationships with both parents and students, how well they understand the content, and how they carry out their teaching. Test scores are only partially influenced by the quality of a teacher and by dismissing faculty based on these scores we are depriving children of role models.