Thursday, December 08, 2011

Do Teachers Really Come From The "Bottom Third" Of College Graduates?

This from the Shanker Blog:
The conventional wisdom among many education commentators is that U.S. public school teachers “come from the bottom third” of their classes. Most recently, New York City Mayor Michael Bloomberg took this talking point a step further, and asserted at a press conference last week that teachers are drawn from the bottom 20 percent of graduates.

All of this is supposed to imply that the U.S. has a serious problem with the “quality” of applicants to the profession.

Despite the ubiquity of the “bottom third” and similar arguments (which are sometimes phrased as massive generalizations, with no reference to actual proportions), it’s unclear how many of those who offer them know what specifically they refer to (e.g., GPA, SAT/ACT, college rank, etc.). This is especially important since so many of these measurable characteristics are not associated with future test-based effectiveness in the classroom, while those that are are only modestly so.

Still, given how often it is used, as well as the fact that it is always useful to understand and examine the characteristics of the teacher labor supply, it’s worth taking a quick look at where the “bottom third” claim comes from and what it might or might not mean.

Most people who put forth this assertion cite one of two sources, both from the McKinsey & Company consulting organization. The first is an influential 2007 report, (based on Condition of Education” (CoE) report, (2002 edition) which simply notes that “we are now recruiting our teachers from the bottom third of high school students going to college.” The authors fail to specify how “bottom third” is defined, or whether their data refer to graduates who planned to teach versus those who actually got a job (the latter method is, of course, far preferable)...

The second standard source for the “bottom third” claims is more clear and well-documented. It is a subsequent McKinsey report (2010), one which doesn’t rely on questionable interpretations from indirect sources, but rather its own analysis (based on SAT/ACT scores, specifically those of 1999 graduates whose first job was teaching). That report claims, “The U.S. attracts most of its teachers from the bottom two-thirds of college classes, with nearly half coming from the bottom third.” ...

Why the differences? Because these studies are looking at different groups of teachers. In the CoE data, it’s 1993 graduates who had taught by 1997 (four years later), while the data used in the second McKinsey include 1999 graduates who, in 2001 (two years later), said their first job was (or is) teaching. In other words, each set of results is based on two different cohorts of college graduates, who are also identified in different ways, at different points after graduation.

Neither sample is necessarily representative of the teacher workforce as a whole, or of prior and subsequent cohorts.

Overall, then, the blanket assertion that teachers are coming from the “bottom third” of graduates is, at best, an incomplete picture. It’s certainly true that, when the terciles are defined in terms of SAT/ACT scores, there is consistent evidence that new teachers are disproportionately represented in this group (see here and here for examples from the academic literature). But the differences are not always as large as is sometimes suggested. They vary by year and sample identification, as well as by other variables, such as school-level characteristics (e.g., poorer schools) and teacher characteristics such as race and gender. (And, by the way, their relative standing as graduates is based on tests that most took in high school.)

Finally, it’s very important to note that the “bottom-“ and “top third” may not be a particularly good conceptualization of either the problem or the solution. The connection between SAT/ACT scores and future (test-based) effectiveness, though it’s among the only associations strong enough to be discerned statistically, is still a highly imprecise predictor of “quality.” There’s some useful information there, but it’s unclear whether it merits the high-profile attention it receives.**

There’s no question that we should try and get as many highly-qualified applicants as possible into the classroom (though, again, how to do so is an open question). But we should also be very careful about oversimplifying the issue of aggregate teacher quality – and making sweeping statements about their qualifications based on a limited body of evidence – for the sake of intuitive, easy-to-understand talking points.


Anonymous said...

Well as someone who would probably be identified as being in the top third, I am wondering where all these low performers are. From what I can recall during my pre-service teaching days, I felt that there were many folks who were quite intelligent, articulate and industrious as students.

I guess my confusion regarding this bottom half assersion about teacher candidates is how in the world did these folks get licensed? If the pre-service teachers are of such a marginal quality, how did they pass state and or PRAXIS exams. Did the colleges do a whiz bang job of preparing them, are the teacher licensing exams too easy or was there some sort miricle whereby they were suddenly graced with superior intellect. Just doesn't stake up saying pre-service teacher pool is populated by poor to marginal students but at the end of their post secondary education they are able to get licenses.

Janea Johnston said...

It seems to me that there is too much of an emphasis on SAT and ACT testing. Are we not learning in the College of Education that not all children do well on standardized tests? Why are we going to judge aspiring teachers the same way? Quality of work should be judged because there are many factors that play into testing. It is surprising to me that people would use this date to say that teachers come from the lower third of their class. Whether someone is great at taking tests or not does not determine how great of a teacher they will be. I just feel like we have learned better than to judge students solely on their test scores.

Tyler Jones said...

Looking at these studies which indicate that teachers generally fall under the bottom third of college students in terms of gpa and class standing, it is important to understand that this does not indicate all teachers fall under this category. Generally speaking, teaching programs house a very large number of students, and because of that it is only natural with such volume that the average student would fall in these ranges ( bottom 2/3rds as one of the studies suggested ), based purely upon the laws of averages, inevitably making high averages for such a large programs nearly impossible. That said, it is also important to understand that great teachers can still arise from these "low standings". As previous bloggers have mentioned, a great amount of emphasis is placed upon the ACT or SAT to rank these students, though these tests in no way indicate how effectively a person can learn or teach a classroom for that matter. I think this is important to understand when looking at these seemingly dark studies towards education.