Education’s Merchant of Doubt

One man’s deceitful mission to undermine fair and adequate school funding

This from Bruce Baker at School Finance 101:

Back in 2012, I opined: “It is hard to imagine a time in the history of American public education when there has been such a widespread political effort to argue that improving the quality of schools has little or nothing to do with the amount of money spent on public education. That is, that money simply doesn’t matter.”[1] It seemed as though at some point, discourse might begin to turn the corner on this question. That it might become more publicly acceptable and even acceptable in some political circles to acknowledge the relevance of money for improving the quality of schooling, and creating more equitable and adequate schools for achieving modern outcome goals.

But that rhetoric persists as strong as ever both in political circles and in the pseudo-academic policy research which informs that rhetoric. Further, even as the economy has begun to rebound state school finance systems have continued to lag, perhaps in part due to the persistent rhetoric regarding the irrelevance of school funding, and preferences for not merely revenue neutral, but revenue negative reforms.

In reference to a legal challenge brought against New York State, by small city school districts, New York’s Governor Cuomo opined:

“We spend more than any other state in the country,”
“It ain’t about the money. It’s about how you spend it – and the results.” [2]

In conversations regarding Federal education spending priorities, Virginia Congressman Dave Brat proclaimed:

“Socrates trained Plato in on a rock and then Plato trained in Aristotle roughly speaking on a rock. So, huge funding is not necessary to achieve the greatest minds and the greatest intellects in history.” [sic][3]

And so it is: we need only provide sufficient collection of rocks to ensure educational adequacy. That is, setting aside the modern-day competitive wage required to recruit and retain philosophy instructors of the quality of Socrates and provide them 1:1 student/teacher ratios.

In recently published analysis, I found that during the recession, state school finance systems took a substantial hit, both in terms of total state and local revenue and in terms of equity between districts serving lower and higher poverty student populations:

The recent recession yielded an unprecedented decline in public school funding fairness. Thirty-six states had a three year average reduction in current spending fairness between 2008-09 and 2010-11 and 32 states had a three year average reduction in state and local revenue fairness over that same time period. Over the entire 19-year period, only 15 states saw an overall decline in spending fairness. In years prior to 2008 (starting in 1993) only 11 states saw an overall decline in spending fairness. [4]

A more recent report from the Center on Budget and Policy Priorities revealed that through 2014-15, most state school finance systems had not yet begun to substantively rebound:

At least 30 states are providing less funding per student for the 2014-15 school year than they did before the recession hit. Fourteen of these states have cut per-student funding by more than 10 percent. (These figures, like all the comparisons in this paper, are in inflation-adjusted dollars and focus on the primary form of state aid to local schools.)

Most states are providing more funding per student in the new school year than they did a year ago, but funding has generally not increased enough to make up for cuts in past years. For example, Alabama is increasing school funding by $16 per pupil this year. But that is far less than is needed to offset the state’s $1,144 per-pupil cut over the previous six years. [5]

In short, the decline of state school finance systems continues and the rhetoric opposing substantive school finance reform shows little sign of easing. Districts serving the neediest student populations continue to take the hardest hit. Yet, concurrently, many states are substantively raising outcome standards for students[6] and increasing the consequences on schools and teachers for not achieving those outcome standards. Some positive signs include recent structural reforms, possibly involving new revenue in California and Pennsylvania, in each case focusing on districts serving high poverty student populations. But other states which cut substantially during the economic downturn, even under the pressure of prior and ongoing judicial review and oversight, have continued to cut (Kansas) or largely freeze state aid (New York).

From the cloud of doubt to a rock of certainty

Eric Hanushek

In my 2012 report Does Money Matter in Education? I explained how one man’s mission to create a cloud of uncertainty surrounding the relationship between school quality and available funding has distorted public policy discourse over school finance reform.

One might characterize Eric Hanushek as education’s own “merchant of doubt.”
I explained the evolution of Eric Hanushek’s frequently reiterated assertions of “no systematic relationship between school expenditures and student performance,” [7] originating in the 1980s, to more recent, bolder claims that substantial funding cuts cause no harm.

While compelling evidence has continued to accumulate regarding the importance of funding for improving school quality, Hanushek in various outlets and public testimony has continued to drift from the cloud of doubt to a rock of certainty. That is, certainty that money has little or no role in improving school quality and that school finance reforms which infuse additional funds only lead to greater inefficiency, having little or no effect on either equity or adequacy of schooling.[8]

To summarize, the current Hanushekian dogma includes the following core principles:

1. Because schools already spend so much and do so with such great inefficiency, additional funding is unlikely (read “will not and cannot”) to lead to improved student outcomes;
2. How money is used matters much more than how much money is spent;
3. Therefore, some schools and districts having more or less than others is inconsequential, since those with less may simply make smarter spending decisions.

According to the recent rhetoric of Hanushek, these principles are ironclad, in in his own words they are “conventional wisdom,” on which “virtually all analysts” agree. They are “commonly believed,” “overall truth,” and backed by an “enormous amount of scientific analysis” and “substantial econometric evidence,” and “considerable prior research.”

For example, in the winter of 2015, in the context of school funding litigation in New York State, Hanushek opined:

“An enormous amount of scientific analysis has focused on how spending and resources of schools relates to student outcomes. It is now commonly believed that spending on schools is not systematically related to student outcomes.”[9]

Yet, the enormous amount of scientific analysis to which Hanushek referred in his expert testimony was primarily cited to a 2003 summary of much of his prior work from the 1980s, work which has been discredited on numerous occasions, [10] not to mention, research that has occurred in the last 12 years.[11] Similarly, in the same context (Maisto v. State) Hanushek proclaims:

“There has been substantial econometric evidence that supports this lack of relationship.”

Backed again (in footnote 6 of his report) by the same short list of dated self-citation.[12] In an even more recent attempt to rebut a new, major study finding positive effects of school finance reforms,[13] Hanushek (2015) makes the following version of the same claim:

“Considerable prior research has failed to find a consistent relationship between school spending and student performance, making skepticism about such a relationship the conventional wisdom.”[14]

This time, anchoring that claim only to his 2003 piece (by hyperlink to the “prior research” phrase) on the Failure of input based schooling policies,[15] choosing to ignore entirely the considerably larger body of more rigorous work I summarize in my 2012 review on the topic.

The extension of these claims that nearly everyone agrees, and all (or, a veritable shit-ton of) research says that there’s no clear relationship between spending and student performance is the assertion that there is broad agreement that how money is spent matters far more than how much there is. As phrased by Hanushek in the context of New York State school finance litigation:

“Virtually all analysts now realize that how money is spent is much more important than how much is spent. This finding is particularly true at the upper levels of current U.S. spending.[16]

As with the prior declarations, this one is made with the exceedingly bold assertion that virtually all analysts agree on this point – without reference to any empirical evidence to that point (a seemingly gaping omission for a decidedly empirical claim about a supposedly empirical truth). Put bluntly, if you don’t have it, you can’t spend it. Thus, the two issues – how much you have and how you spend it – are inextricably linked.
Perhaps most disconcerting is that Hanushek has recently extended this argument to declare that equity gaps in funding, or measures of them, aren’t an important policy concern either. They are, by his proclamation “vacuous” and “lacking any scientific basis.”[17]

Put differently, what Hanushek is opining by declaring calculations of equity gaps to be vacuous and lacking scientific basis is that it matters not whether one school or district has more resources than another. Regardless of any spending differences, schools and districts can provide equitable education – toward equitable outcome goals. Those with substantively fewer resources simply need to be more efficient. Since all public schools and districts are presently so inefficient, achieving these efficiency gains through more creative personnel policies, such as performance based pay, and dismissal of “bad teachers”, are easily attainable.

Of course, even if we assume creative personnel policies to yield marginal improvements to efficiency, if schools with varied levels of resources pursued these strategies with comparable efficiency gains, inequities would remain constant. Requiring those with less to simply be more efficient with what they have is an inequitable requirement. This argument is often linked in popular media and the blogosphere with the popular book and film Moneyball, which asserts that clever statistical analysis for selecting high productivity, undervalued players was the basis for the (short lived) success of the low payroll in 2002 and 2003 Oakland A’s baseball team. The flaws of this analogy are too many to explore thoroughly herein, but the biggest flaw is illustrated by the oft-ignored subtitle of the book – The art of winning an unfair game. That is, gaining a leg up through clever player selection is necessary in baseball because vast wealth and payroll differences across teams make baseball an unfair game. Put bluntly, public schooling should not be an unfair game.

The Eroding Soil under the Rock

From judges to scholars, critics of evidence (other than myself) used by Hanushek to support the above claims have characterized that evidence as “facile,” based on “fuzzy logic”[18] and “weak and factually tenuous.”[19]

Two recurring examples used by Hanushek to illustrate the unimportance of funding increases for improving outcomes, are the “long term trend” or “time trend” argument, and anecdotal claims of the failures of input-based reforms in New Jersey. Baker and Welner (2011) tackle in depth, the fallacies of Hanushek’s New Jersey claims.[20] Here, I point to Hanushek’s own, albeit facile, unacknowledged self-debunking of his New Jersey claims. But first, I address the “long term trend” claim.

Again from recent testimony in New York State, Hanushek provides the following exposition of the “long term trend” assertion:

The overall truth of this disconnect of spending and outcomes is easiest to see by looking at the aggregate data for the United States over the past half century. Since 1960, pupil‐teacher ratios fell by one‐third, teachers with master’s degrees over doubled, and median teacher experience grew significantly (Chart 1).4 Since these three factors are the most important determinants of spending per pupil, it leads to the quadrupling of spending between 1960 and 2009 (after adjusting for inflation). At the same time, plotting scores for math and reading performance of 17‐year‐olds on the National Assessment of Educational Progress (NAEP, or “The Nation’s Report Card”) shows virtually no change since 1970 (Charts 2 and 3).5[21]

This claim like many others is made with language of astounding certainty – the “overall truth” as it exists in the mind of Hanushek. This claim is commonly accompanied by graphs showing per pupil spending going up over time, pupil to teacher ratios going down, and national assessment scores appearing relatively flat, much of which is achieved via the smoke and mirrors of representing spending and outcome data on completely different scales, and failures to adjust appropriately for changing costs and related obligations of the public education system, and changing demography of the tested population.[22] Oversimplified visuals are used to make the proclamation that student achievement shows “virtually no change,” a statement discredited on closer inspection.[23] Jackson and colleagues provide additional examples of how such facile analyses lead to fallacious conclusions (ironically using cigarette smoking data).[24]
Hanushek extends his use of the long term trend argument in his recent critique of findings from Jackson and colleagues that court ordered infusions of funding to select schools and districts led to long term gains in educational attainment, income and poverty reduction for those subjected to increased funding. Hanushek asserts:

If a ten percent increase yields the results calculated by Jackson, Johnson, and Persico, shouldn’t we have found all gaps gone (and even reversed) by now due to the actual funding increases?

Thus, if the massive average spending increases reported by Hanushek as the actual long term trend did not lead to elimination or reversal of gaps, Jackson, Johnson and Persico’s findings must be wrong? Right?

Of course, this assertion is complete and utter nonsense, because Jackson and colleagues don’t assert, and Hanushek’s own national average long term trend data do not show that all low income children, lower performing subgroups and/or those in low wealth communities were subjected to dramatic funding increases. In fact, if Hanushek’s average spending increases were driven as much by increases in wealthy (low poverty/minority) districts as they were by increases in poorer districts, then gaps would likely remain constant, all else equal.  That is, the average level of funding, and changes in average level say nothing of gaps or distributions in funding or changes in gaps or distributions. Put bluntly, the average level of funding, and the distribution of funding are two different things. Conflating the two is intentionally deceitful.

As explained by Baker and Welner (2011)[25] Hanushek for years has cited the failures of New Jersey’s school finance reforms as the basis for why other states should not increase funding to high poverty schools. In litigation in Kansas in 2011, Hanushek proclaimed:

“The dramatic spending increases called for by the courts (exhibit 34) have had little to no impacts on achievement. Compared to the rest of the nation, performance in New Jersey has not increased across most grades and racial groups (exhibits 35-40). These results suggest caution in considering the ability of courts to improve educational outcomes.”[26]

Hanushek reiterated these claims in the context of the even more recent New York school funding challenge. [27] This is a surprising claim to preserve when one’s own recent (2012) marginally more rigorous analyses of state achievement growth rates on national assessments (from 1992 to 2011)[28] find the following:

“The other seven states that rank among the top-10 improvers, all of which outpaced the United States as a whole, are Massachusetts, Louisiana, South Carolina, New Jersey, Kentucky, Arkansas, and Virginia.”[29]

The same report by Hanushek shows impressive reductions in the share of students scoring “below basic” in New Jersey, especially for 8th grade math (Figure 4).
To be sure, there are others in academe and policy research that raise questions about the most effective ways to leverage school funding to achieve desired outcomes, and do so via more rigorous, thoughtful analyses.

There are others who opine in the public square[30] and courtroom[31] that school finance reform – specifically infusing additional funding to districts serving high need student populations – is neither the most effective nor most efficient path toward improving schooling equity or adequacy. But empirical evidence to support claims of more efficient alternatives remains elusive.

Nonetheless, the “facile” and “factually tenuous” illustrations above must be put to rest, and the divisive, manipulative (intellectually insulting) and damaging rhetoric of education’s merchant of doubt cast aside once and for all.

NOTES
[1] Baker, B. D. (2012). Revisiting the Age-Old Question: Does Money Matter in Education?. Albert Shanker Institute.
[2] http://blogs.wsj.com/metropolis/2014/02/11/cuomo-on-education-funding-lawsuit-it-aint-about-the-money/
[3] http://thinkprogress.org/education/2015/02/13/3623158/brat-education-plato/
[4] Baker, B. D. (2014). Evaluating the recession’s impact on state school finance systems.
Education Policy Analysis Archives, 22(91). http://dx.doi.org/10.14507/epaa.v22n91.2014
[5] Leachman, M., & Mai, C. (2014). Most States Still Funding Schools Less Than Before the Recession. Center on Budget and Policy Priorities, October 16, 2014, http://www. cbpp. org/cms/index. cfm? fa= view&id, 4213.
[6]
Bandeira de Mello, V., Bohrnstedt, G., Blankenship, C., and Sherman, D. (2015). Mapping State Proficiency Standards Onto NAEP Scales: Results From the 2013 NAEP Reading and Mathematics Assessments (NCES 2015-046). U.S. Department of Education, Washington, DC: National Center for Education Statistics. Retrieved [date] from http://nces.ed.gov/pubsearch.
[7] Hanushek, E.A. (1986) Economics of Schooling: Production and Efficiency in Public Schools. Journal of Economic Literature 24 (3) 1141-1177. A few years later, Hanushek paraphrased this conclusion in another widely cited article as “Variations in school expenditures are not systematically related to variations in student performance”
Hanushek, E.A. (1989) The impact of differential expenditures on school performance. Educational Researcher. 18 (4) 45-62
Hanushek describes the collection of studies relating spending and outcomes as follows:

“The studies are almost evenly divided between studies of individual student performance and aggregate performance in schools or districts. Ninety-six of the 147 studies measure output by score on some standardized test. Approximately 40 percent are based upon variations in performance within single districts while the remainder look across districts. Three-fifths look at secondary performance (grades 7-12) with the rest concentrating on elementary student performance.” (fn #25)

[8] Notably, Hanushek then and now asserts that it’s not that money doesn’t matter at all, but rather that additional money doesn’t matter on top of the already high (apparently indisputably and invariably) levels of spending that currently exist across all U.S. schools.
[9] http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State%27s%20Expert%20Report%20-%20Dr.%20Eric%20Hanushek.pdf
[10] Baker, B. D. (2012). Revisiting the Age-Old Question: Does Money Matter in Education?. Albert Shanker Institute.
[11] Including but not limited to:
Jackson, C. K., Johnson, R., & Persico, C. (2015). The Effects of School Spending on Educational and Economic Outcomes: Evidence from School Finance Reforms (No. w 20847) National Bureau of Economic Research.
Papke, L. (2005). The effects of spending on test pass rates: evidence from Michigan. Journal of Public Economics, 89(5-6). 821-839.
Hyman, J. (2013). Does Money Matter in the Long Run? Effects of School Spending on Educational Attainment. http://www-personal.umich.edu/~jmhyman/Hyman_JMP.pdf.
Deke, J. (2003). A study of the impact of public school spending on postsecondary educational attainment using statewide school district refinancing in Kansas, Economics of Education Review, 22(3), 275-284. (p. 275)
Nguyen-Hoang, P., & Yinger, J. (2014). Education Finance Reform, Local Behavior, and Student Performance in Massachusetts. Journal of Education Finance, 39(4), 297-322.
Downes, T. A. (2004). School Finance Reform and School Quality: Lessons from Vermont. In Yinger, J. (Ed.), Helping Children Left Behind: State Aid and the Pursuit of Educational Equity. Cambridge, MA: MIT Press
[12] Specifically, Hanshek includes the following footnote:

Hanushek (2003). See also Hanushek (1981, (1986, (1989). The statistical analyses focus on the independent impact of resources on performance after allowing for differences among families, peers, and neighborhoods. A variety of sophisticated approaches have been applied to schooling situations across the countries, and the reviews summarize these studies. The aggregate results of the most sophisticated of these studies are shown below.

[13] Jackson, C. K., Johnson, R. C., & Persico, C. (2015). The effects of school spending on educational and economic outcomes: Evidence from school finance reforms (No. w20847). National Bureau of Economic Research.
[14] http://hanushek.stanford.edu/opinions/does-money-matter-after-all
[15] http://hanushek.stanford.edu/sites/default/files/publications/Hanushek%202003%20EJ%20113%28485%29.pdf
[16] http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State%27s%20Expert%20Report%20-%20Dr.%20Eric%20Hanushek.pdf
[17] Specifically, Hanushek proclaims:

It also underscores how calculations of equity gaps in spending, of costs needed to achieve equity, or of costs needed to obtain some level of student performance are vacuous, lacking any scientific basis.” (Maisto, p4)

http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State%27s%20Expert%20Report%20-%20Dr.%20Eric%20Hanushek.pdf
[18] http://educationnext.org/money-matter/
[19] http://www.shawneecourt.org/DocumentCenter/View/457
[20] Baker, B., & Welner, K. (2011). School finance and courts: Does reform matter, and how can we tell. Teachers College Record, 113(11), 2374-2414.
[21] http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State’s%20Expert%20Report%20-%20Dr.%20Eric%20Hanushek.pdf
[22] http://junkcharts.typepad.com/junk_charts/2011/04/bill-gates-should-hire-a-statistical-advisor.html
[23] See, for example:
http://www.epi.org/publication/fact-challenged_policy/
[24] The authors explain:

To see the problems of Hanushek’s logic, consider the following true statistics: between 1960 and 2000 the rate of cigarette smoking for females decreased by more than 30 percent while the rate of deaths by lung cancer increased by more than 50 percent over the same time period.[1] An analysis of these time trends might lead one to infer that smoking reduces lung cancer. However, most informed readers can point out numerous flaws in looking at this time trend evidence and concluding that “if smoking causes lung cancer, then there should have been a large corresponding reduction in cancer rates so that there can be no link between smoking and lung cancer.” However, this is exactly the facile logic invoked by Hanushek regarding the effect of school spending on student achievement.

http://educationnext.org/money-matter/
[25] Baker, B., & Welner, K. (2011). School finance and courts: Does reform matter, and how can we tell. Teachers College Record, 113(11), 2374-2414.
[26] http://www.robblaw.com/PDFs/1169.pdf
[27] http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State’s%20Expert%20Report%20-%20Dr.%20Eric%20Hanushek.pdf
[28] As explained by the authors:

We also examine changes in student performance in 41 states within the United States between 1992 and 2011, allowing us to compare these states with each other.
Our findings come from assessments of performance in math, science, and reading of representative samples in particular political jurisdictions of students who at the time of testing were in 4th or 8th grade or were roughly ages 9–10 or 14–15.

[29] Hanushek, E. A., Peterson, P. E., & Woessmann, L. (2012). Is the US catching up: international and state trends in student achievement. Education Next, 12(4), 24.
http://www.hks.harvard.edu/pepg/PDF/Papers/PEPG12-03_CatchingUp.pdf
[30] http://jaypgreene.com/2015/05/29/does-school-spending-matter-after-all/
[31]Including renowned segregationist David Armor who continues to testify alongside Hanushek, http://www.edlawcenter.org/assets/files/pdfs/maisto/masito%20trial%20documents/State’s%20Expert%20Report%20-%20Dr.%20David%20Armor.pdf