For international readers, I would like to summarize a piece on false economic research supporting tuition fees, which appeared in German in Handelsblatt on 19 February. As interesting as the fake research itself is the differing reactions of the two main channels, which had been used to publicize it: One was the prestigious Working Paper series of the National Bureau of Economic Research in the US The other was the well-read platform Vox (voxeu.org) of the London-based Centre for Economic Policy Research.
Britain started to impose tuition fees starting in 1998. From 1000 GBP, at first just for the wealthier students, the fees went up steeply to 9250 GBP for all students today. On average, graduates leave university with about 50.000 GBP in debt. With Jeremy Corbyn rather successfully campaigning against tuition fees and one of the architects of the current system, former Blair advisor Andrew Adonis demanding its abolition, the system has come under fire in Britain. The same is true for the US. Critics assume that there is a deterring effect of high tuition fees on young people from disadvantaged households.
A surprising finding
However, a widely publicized study by three economists came to the conclusion that the British system of tuition fees has improved enrollment numbers for young people from disadvantaged households – both in absolute terms and relative to those from wealthier households. Richard Murphy (U. of Texas, Austin), Judith Scott-Clayton (Columbia) and Gillian Wyness (University College London) made the claim of improved equity in their NBER Working Paper The End of Free College in England: Implications for Quality, Enrolments, and Equity, published in September 2017.
The conclusion is false, though. They claim that the enrollment-rate of the most disadvantaged households rose the most from 1997 to 2015 and that it doubled in that period to 20 percent. The corresponding graph of the data, however, shows only a small increase from about 11 to about 13 percent in the bottom quintile. The increase in enrollment of the middle 60 percent of the distribution is more pronounced.
The same faulty conclusion is contained in a summary-piece on Vox, dated 21 October. It was read more than 12,000 times.
I pointed the error out to the authors. One of them, Wyness, admitted the mistake and explained that the numbers referred to a different graph, which had inadvertently been left out. She pointed me to a weblog article from December, in which she supports the claim with more evidence. Wyness also declared upon request that there was no need to correct the factual mistake in the other publications, as it was only preliminary research. A number of media had, however, reported on this study in the context of the political discussion about tuition fees, even including German media.
On inspection, it turned out that Wyness’ description of the additional evidence provided in the post on the specialized weblog WonkHE was (and still is) again highly misleading. She presents a graph of enrollment rates over time by regions divided into quintiles according to regional enrollment rates. Text and headline, however, falsely represent the data as pertaining to quintiles of households. Only if you know what “POLAR3” means, or look it up in the source report, will you be able to find out that you have been misled. The increase of enrollment from 10 to 20 percent is there to see in the graph, but it refers to a different time-frame and to regions instead of households.
Since this started to appear like intentional misleading, I asked CEPR-director Richard Baldwin, editor-in-chief of Vox, and James Poterba, president of NBER, about their policies regarding correction or retraction of erroneous or fraudulent preliminary research published on their publication channels. I explained the problems with the tuition paper. Baldwin reacted with a request for further explanation and then did not answer further requests for comment. Poterba reacted swiftly. He alerted the authors to the possibility of posting revised working paper versions in the NBER series, which they did in January. The misleading statements on Vox, in contrast, have not yet been corrected or retracted, nor has the Wyness blogpost on WonkHE.
The revised paper was faulty again
The revised NBER Working Paper was, however, misleading in the relevant section again. The authors corrected the false description of the household time-series. They proceeded by saying that this data was not very reliable. They added the graph from the Wyness-blog-post depicting the data by regions. As Wyness had done, they presented the regional data as if it was household data. Only in a footnote they correctly explained, that it was not really household data (and that by implication, the household-terminology used in the main text was wrong).
I alerted James Poterba to this discrepancy and to the fact that the unsubstantiated claim that equity had improved was still in the abstract of the paper. He had the authors provide a second revision, which is the one, which is currently on-line. In this revised version, they clearly stated that the additional data was by region only. However, despite not providing any evidence or rationale for concluding that narrowing enrollment differences by region would imply narrowing differences by parental income, the conclusion in the abstract remained unchanged:
“We conclude that tuition fees, at least in the English case supported their goals of increasing quality, quantity, and equity in higher education.
Conclusion: The editor-in-chief of CEPR’s Vox does not seem to mind if authors mislead the public with false research results via this influential channel. The president of NBER seemed concerned that obvious errors are corrected, though not enough to make sure that unsupported conclusions are also corrected. As regards the authors: they give the impression that they care a lot more about the financial situation of universities, than about an objective answer to their research question.
[20.2.2017]
