Initial analysis of the Organisation for Economic Cooperation and Development’s (OECD) Programme for International Student Assessment (PISA) suggests that mobile phone bans might help with reducing student distraction in school. However, our examination of student outcomes and phone bans has revealed an inverse relationship: the more a country bans phones, the lower their PISA score. When controlling for gender, school behaviour and socio-economic status, phone bans are still negatively associated with PISA results in OECD countries. This blog post examines the relationship between achievement scores on PISA tests and mobile phone bans in schools.

Banning mobile phones in schools

The Secretary of State for Education’s recent announcement that mobile phone use is to be banned in English schools (DfE, 2023) has stoked a debate about the value and practicality of such bans. A source of data on the effectiveness of mobile phone bans is from the OECD’s 2022 PISA data set – a triennial international assessment. Nearly 700,000 students in 81 OECD member countries participated in the 2022 survey (OECD, 2023a). During the launch event of the 2022 UK PISA report, one of the authors of that report commented that ‘the only policy negatively related to [student] distraction is banning mobile phones’ but added there was not enough evidence to warrant a complete ban (Policy Exchange, 2023). While the relationship between distraction and phone use is interesting, we examined PISA 2022 data to explore the relationships between academic achievement and phone bans in schools.

‘Our examination of student outcomes and phone bans has revealed an inverse relationship: the more a country bans phones, the lower their PISA score.’

The impact of mobile phone bans on achievement

The PISA 2022 survey (OECD, 2023b) contains an item in the school questionnaire in which administrators report a ban through a ‘Yes/No’ response. Reports of bans through this item do not allow schools to report partial bans. With this caveat, when the percentage of schools that ban phones in a country is plotted against mean results in science, mathematics and reading for that country (see figure 1), a statistically significant (p < .001***, R ² = .13) but small negative trend exists, shown by the blue line on the plot. For every 10 per cent increase in the number of schools in a country banning mobile phones, mean PISA scores fall by 0.09 of a standard deviation, or 9.4 points. This means that, in general, the higher the percentage of schools in a particular country that have bans in place for mobile phones, the lower that country’s average PISA score. Additionally, OECD countries appear to perform very differently from non-OECD countries, getting higher science, mathematics and reading results on average, and being less likely to implement bans.

Figure 1: Normalised mean country score for maths, reading and science against percentage of schools banning phones. Selected outlying countries, and the UK, labelled.

Possible explanations for this negative association may be related to student behaviour, socio-economic status or gender. To further explore this hypothesis, we fitted a linear model to data from OECD countries only, predicting student mean scores for mathematics, reading and science. When student gender, index of economic, social and cultural status (ESCS), and school behaviour are controlled for, phone bans still have a statistically significant impact on standardised PISA results (p < .001***, R ² = .20), albeit a small one. Banning phones on school premises explains students getting 9.1 points lower mean mathematics, reading and science scores, or 0.10 of a standard deviation lower.

Table 1: Linear model – Mean student PISA score standardised by country, predicted by standardised index of economic, social and cultural status (ESCS), student gender, school phone ban and standardised school behaviour

Note: This table reports the impact of different factors on the average student PISA score within a country. Each factor reports a p-value, whether the factor is statistically significant, and a Beta value, the number of standard deviations difference in the PISA score resulting from a change in this factor, along with the 5–95% confidence interval for that Beta value. All the factors were calculated as statistically significant (<0.001). Overall, the model was also calculated to be statistically significant.

Among the UK regions, only in England were phone bans statistically significant, with schools applying bans seeing significantly worse results (p < .001***, R² = .15), a typical fall of 0.29 of a standard deviation, compared to those without.

Conclusions

In the launch of the 2022 survey data in the UK, the OECD, while somewhat equivocal, presented evidence in favour of banning mobile phones, based on student distraction data. By contrast, we present a curious and contradictory finding that, when gender, social class and school behaviour are controlled for, students in schools with phone bans have lower achievement across their PISA test scores than those in schools that allow phone use. Though it should be noted that in all of our models, the effect sizes, R², and Beta values are low, and other unexplored factors might be more useful in explaining student outcomes. Our conclusion is that when considering a mobile phone ban, the relationship between a range of variables – not just student distraction – should be investigated to support policymakers in deciding to implement phone bans in schools.

References

Department for Education [DfE]. (2023, October 2). Mobile phone use to be banned in schools in England [Press release]. https://www.gov.uk/government/news/mobile-phone-use-to-be-banned-in-schools-in-england.

Policy Exchange. (2023, December 5). Launch of the UK PISA 2022 results. https://policyexchange.org.uk/events/launch-of-the-uk-pisa-2022-results/

Organisation for Economic Co-operation and Development [OECD]. (2023a). Decline in educational performance only partly attributable to the COVID-19 pandemic. https://www.oecd.org/newsroom/decline-in-educational-performance-only-partly-attributable-to-the-covid-19-pandemic.htm  

Organisation for Economic Co-operation and Development [OECD]. (2023b). PISA 2022 results: Preparing students for a changing world. https://www.oecd.org/publication/pisa-2022-results/