Blog post
Assessing the impact of STEM initiatives in improving educational outcomes – have they worked?
The general trend for science and maths A-level entries as a percentage of all A-level entries has been downwards over the last 15 years. Research evidence from large scale national surveys conducted in the UK on 10-14 year olds reported most students enjoyed learning science in school but did not consider pursuing it as a career (DeWitt, J. & Archer, L. 2015). A strong correlation has been shown between interest, attitudes and aspirations towards STEM subjects and the take-up of these courses (Banerjee, 2016b).
To help young people develop a positive attitude towards science, technology, engineering and mathematics (STEM) early-on the formal and informal education sector worked together. The focal point for these efforts has been the 11-16 age group – a critical period for forming views on STEM and developing aspirations. STEM enrichment and enhancement activities have thus been delivered across primary and secondary schools. The activities include science practical lessons, supported by ambassador visits, trips to laboratories, STEM centres and higher education institutions. The common theme for these activities has been their aim to improve student understanding and enjoyment of science in the short-term and encourage STEM participation in the long-term.
A longitudinal national evaluation of STEM ‘enrichment and enhancement activities’ was conducted making use of school and pupil level data from the national pupil database (NPD)(Banerjee, 2016). The three main educational outcomes considered to assess the impact of these STEM enrichment and enhancement activities were – a) school performances in terms of percentage of pupils achieving a ‘good’ GCSE grade in science and maths, b) GCSE attainment in science and maths of pupils and c) continued post-16 STEM participation.
First, analysis of school level data did not show a direct impact of these activities on improved school GCSE performances (Banerjee, 2015; Banerjee, 2017b).
Second, the 2007 cohort across all state maintained secondary schools in England was followed up from the beginning of key stage 3 to the end of key stage 5. Effect size estimates and regression models showed GCSE attainment of pupils in science and maths is not affected by participation in these activities. If pupil background characteristics such as eligibility for free school meals (FSM), prior attainment, gender, ethnicity, speaking English as an additional language (EAL) and disability (SEN) are known it is possible to predict the attainment of students with nearly 80% . It does not matter whether the students actively registered for these activities.
Third, the study investigated whether engaging in these STEM programmes, run for 11 to 16 year olds, in secondary school is likely to affect subject choices made by these pupils during post compulsory education? A direct noticeable impact of these activities was not seen on STEM take-up (Banerjee, 2017a). There is no evidence to suggest continued engagement in these activities is manifested in terms of increasing or widening STEM participation. Post-16 participation estimated for AS and A levels showed young people sparsely represented in STEM courses such as those from a lower socio economic class (identified in NPD by their eligibility for free school meals) and black ethnic minority did not engage any better with STEM subjects following active continued participation in these schemes (Banerjee, 2017a).
Given the huge resources involved in the delivery of these initiatives there is a need to identify what works and if any of these schemes work better?
Publications from the project linked to this write up
Banerjee, P. A (2017a). Is informal education the answer to increasing and widening participation in STEM education? Review of Education (forthcoming)
Banerjee, P. A (2017b). Does continued participation in STEM enrichment and enhancement activities affect school maths attainment? Oxford Review of Education, 43(1) http://dx.doi.org/10.1080/03054985.2016.1235031
Banerjee, P.A. (2016a) A longitudinal evaluation of the impact of STEM enrichment and enhancement activities in improving educational outcomes. International journal of educational research, 76 (1), pp. 1-11. http://dx.doi.org/10.1016/j.ijer.2015.12.003
Banerjee, P. A. (2016b). A systematic review of factors linked to poor academic performance of disadvantaged students in science and maths in schools. Cogent Education, 3(1), 1178441. https://doi.org/10.1080/2331186X.2016.1178441
Banerjee, P. A (2015). Can schemes to inspire tomorrow’s scientists close the poverty attainment gap? The Conversation https://theconversation.com/can-schemes-to-inspire-tomorrows-scientists-close-the-poverty-attainment-gap-35805
DeWitt, J. & Archer, L. (2015) Who Aspires to a Science Career? A comparison of survey responses from primary and secondary school students, International Journal of Science Education, 37:13, 2170-2192, DOI: 10.1080/09500693.2015.1071899