Recently research sponsored by the Independent Schools Council (ISC) was released that “attending an Independent school in England is associated with the equivalent of two additional years of schooling by age 16”. On the surface this research appears rigorous, the authors claim to have controlled for the different background characteristics, and prior attainment, of the pupils attending the different schools and to still find large benefits associated with private school. However, there are major flaws in the method used by the researchers and as a result the study cannot be used as evidence to show the effectiveness of independent schools.

The biggest issue is the use of aggregate school level results, not the individual results of the pupils within the school. This is important because the study claims to show private schools are better at an individual pupil level. However using the aggregate level data for this is inappropriate as decades of research by Harvey Goldstein and others has shown that “such analysis tells us nothing about the effect on individual students”, which is what we need to know about if we are to understand if one type of school is more effective than another. To explain why let’s take two hypothetical schools, school A is a state school and school B is a private school. School A has a wide ability intake while school B’s intake is on average higher attaining, and the pupils are more tightly clustered around this average. The relative average attainment of the schools matters because English GCSE data shows that a greater proportion of higher attaining pupils make more progress between assessments than their peers with lower levels of prior-attainment. It would therefore be possible for all pupils in School A to be making more progress than equivalent ability pupils in school B, but an overall school average would show School A making less progress than school B, simply because School B has a larger proportion of higher prior-attaining pupils.

If private schools have an intake with a greater proportion of already high attaining pupils compared to state schools, then the model used in the Durham study will show the private schools making more progress, even though in reality the state schools are doing better with every level of pupil ability.

This kind of distribution is exactly what is shown in the data of the Durham study, state school intakes are shown as having lower average prior attainment and larger standard deviations. A clear sign state schools are not only educating children who are on average attaining lower than those in private schools, but also have a wider range of abilities.

The use of linear regression modelling is another issue. This method can estimate the individual contribution of various factors on an outcome but cannot allow us to see how these variables interact with each other. This is important because the state schools in the sample are more deprived and lower attaining, so we don’t just want to see the contribution of disadvantage or low attainment on progress, we need to know if being disadvantaged and low achieving affects individual pupils differently than one would expect from just adding the individual effects together. Amongst school effectiveness researchers it is well established that “these factors need to be taken into account if we wish to make useful, causally connected, inferences about the effects of schools on progress.”

the research tells us nothing beyond the fact that private and state schools have different intakes

The methods chosen by the Durham researchers are therefore inappropriate for research into the relative effectiveness of different school types. As a result the research tells us nothing beyond the fact that private and state schools have different intakes, and it certainly does not say that private schools are adding more value than state schools.

In truth the most interesting thing about the research is that it was commissioned at all. Independent schools have traditionally used simplistic analysis to justify their fees. However as state school results have been improving, and as the narrative around state schools has begun to be more positive across publications of all political stripes, the schools represented by the ISC are probably needing to work harder to recruit pupils, hence the need for this research. In this light, the study is less a troubling revelation that English private schools continue to provide unfair advantages to wealthy children, and instead might be a sign that they’re beginning to struggle. Indeed this may be an international pattern, with PISA finding that across the OECD, public and private schools with comparable student populations performed the same.