Higher Education Policy Institute report on access: the debate rages on

15 January 2019

By Lindsey Macmillan, Matt Dickson, Simon Burgess

We appreciate the response by Iain Mansfield on WonkHE to the widespread criticism of his paper on selective schooling. However, the points we made about the dataset used and the methods employed remain.

A major critique that has yet to be answered is the inappropriate comparisons made when analysing progression to HE. The key part of the argument about the effectiveness of selective schools is hinged on analysis that is far too simple to support the strong statements made. Mansfield returns to the 39% vs 23% rates of progression from selective compared to non-selective areas in his response. The fact that he again attributes these large differences in progression rates directly to the schooling systems, rather than other factors involved that muddy the waters, is a basic stats mistake. The comparison group of all non-selective areas is wrong. If instead the non-selective group of areas were chosen with similar characteristics to the selective areas, the progression rate would be much closer to the 39% rate for selective areas (we will soon be publishing a working paper that shows this is the case).
The point is that the HEPI report attributes all of the difference in selective HE participation rates to the schooling system, while ignoring completely the very significant basic differences between people living in different areas (which also drive different participation rates). Correlation is not causation.
When you make the statistically-right comparison, the progression rates are very similar between the two types of schooling system. There are non-selective areas that are just as good as selective areas at getting this type of progression. Yes, they have particular characteristics, but that is very much the point. It’s those characteristics, rather than the systems per se driving this high participation rate.
The point on the data is that the sample is far from representative – it is missing a lot of high earners. This is illustrated by 70% of the sample falling below the national median figure. Even though median income is defined within sample for the analysis, the fact that the dataset is missing a whole chunk of self-employment data makes any stat about the income profile misleading. Consider the case where one parent runs a company and the other makes a small sum of money in a part-time job. The family will be defined as low income in this data based on the earnings of the parent who makes a small sum of money, rather than reflecting the full sum of income available from the CEO parent. The report uses the figure that 45% of pupils come from families below median household income to conclude that grammars offer wide access to ‘ordinary working families’ and therefore increase social mobility – it’s mentioned a number of times in that way – but this is simply misleading and unreliable given the significant issues with the dataset.
Finally, the emotional argument about “not holding other children back” shows an astonishing lack of perspective on a selective system. It completely ignores the 80% of children that don’t go to grammars, many of whom are dramatically “held back”. What if the child of the warehouse supervisor, the immigrant parent or the shop assistant is one of the majority who miss out on a grammar school (as is far more likely to be the case based on more reliable analysis of the backgrounds of grammar school pupils)? As has been shown before, those who live in selective areas but don’t get into the grammar school suffer significant penalties in terms of education and wages.
Emotional arguments might appeal to the masses but they are not a good basis for making policy decisions. That should be based on evidence from sound analysis of high-quality data. The real reason why academics are united against grammar schools is that the evidence that they damagesocial mobility is robust and conclusive.
Taking all these points together, this discussion continues to be a distraction from genuinely improving the life chances of kids from disadvantaged backgrounds. Anyone who seriously wants to do that knows that grammars are not the answer.