In 2017, the Office for Standards in Education, Children’s Services and Skills in England (Ofsted) stated for the first time that all of its work is evidence-led. Specifically, the regulator’s ‘work should be evidence-led and evaluation tools and frameworks should be as precise, valid and reliable as possible’ (Ofsted, 2017, p. 6). This followed on from the findings and recommendations of a Department for Education in England white paper, which set out an approach to achieving ‘Educational Excellence Everywhere’, and a two-year study to assess progress towards evidence-informed teaching in England.

These findings were advocated in my previous blog’s call for teachers and school leaders to have essential knowledge of evidence-based research, which continues to be Ofsted’s priority: ‘Our policies, frameworks, judgements and insights are rooted in evidence’ (Ofsted, 2022, p. 5). In its latest strategy report, Ofsted states that it is guided by principles ‘grounded in what works best to improve outcomes for children and learners’ and that it understands ‘the perverse incentives and unintended consequences their work can have’ (Ofsted, 2022, p. 4).

There has been a shift in Ofsted’s principles over the years as Coffield (2017) concluded in a BERA Blog post that prior to 2017 Ofsted did ‘more harm than good. Its methods, although changed every few years during the 25 years of Ofsted’s existence, were invalid, unreliable and unjust.’ So, what was happening in practice prior to 2017 and what do we have to watch out for in schools today?

Simply put, not all data gathering (research methods) were valid. Validity is defined by Ofsted as ‘the extent to which judgements assess what they are supposed to’ (Finch, 2019). Measuring a child’s knowledge for maths for example, according to Ofsted, would involve the child attending school for the maths lessons before assessing their understanding using a summative assessment; as opposed to testing what they are yet to be introduced to. Hence, giving an end-of-term summative assessment test at the beginning of the term before the children have covered the content, is using a test (summative assessment) against its purpose – it is not assessing what it is designed to do. This form of data gathering is not valid and is against Ofsted’s principles. Furthermore, such a test should not be the only form of assessment but rather would be supplemented by observations, work samples and discussions with the pupil.

Such practices in primary schools directly relate to the perverse incentives and unintended consequences that Ofsted refer to. For such invalid data-gathering practices often set the child up to fail and can be harmful. An example is baseline assessments that are often used due to the pressures placed on teachers to perform in terms of pupil progress. Baseline assessments were identified as problematic in recent research by Lynch (2022) where teachers revealed that tests sat when returning from Covid-19 restrictions were detrimental to the students’ wellbeing.

If a child shows limited understanding through observations, work samples and discussions, then why would there be a need to use a summative assessment test and risk embarrassment? This is often a result of limitations in teachers’ and school leaders’ data-gathering (research methodology) knowledge. Hence, it is important that such assessments are identified and stopped so children’s confidence in their learning can develop and not be damaged.

‘It is important that invalid summative assessments are identified and stopped so children’s confidence in their learning can develop and not be damaged.’

It has been argued that invalid summative assessments used as baselines set the child up to initially fail in their first test: ‘schools have an incentive to keep Baseline results low in order to show more progress later on’ (Bradbury et al., 2018, p. 6). Hence, initial tests often indicate that the child is actually regressing rather than the assessment’s intention of showcasing progress, ‘leaving schools at the mercy of unreliable data’ (Bradbury et al., 2018, p. 5). It is important that teachers and school leaders are made aware of the significance of data-gathering strategies and the powerful messages they send young minds: ‘Baseline testing is deeply flawed. It will damage children and schools’ (Bradbury et al. 2018, p. 14).

All schools in England need to continue to move forwards with Ofsted and evidence-based practice. This requires all teachers and especially school leaders to have an understanding of research methods and the hidden messages they send to young minds (Lynch, 2022). Everyone who works with children and parents cannot be expected to be research experts, but they can consider the implications of their actions – if it doesn’t feel right then refrain from doing it.

References

Bradbury, A., Jarvis, P., Nutbrown, C., Roberts-Holmes, G., Stewart, N., & Whitebread, D. (2018). Baseline assessment: Why it doesn’t add up. More Than a Score. Baseline_assesment_doc_v1.indd (morethanascore.org.uk)

Coffield, F. (2017, November 2). The research evidence for and against Ofsted. BERA Blog. https://www.bera.ac.uk/blog/the-research-evidence-for-and-against-ofsted

Finch, A. (2019, August 20). The nuance of reliability studies. Ofsted blog. https://educationinspection.blog.gov.uk/2019/08/20/the-nuance-of-reliability-studies/

Lynch, T. (2022). Leading school recovery from the impact of Covid-19: Two birds, one stone. Education 3-13. Advance online publication.  https://doi.org/10.1080/03004279.2022.2068638

Ofsted. (2017). Ofsted strategy 2017–22. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/648212/Ofsted_strategy_2017-22.pdf  

Ofsted. (2022). Ofsted strategy 2022–27. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1070946/Ofsted_Strategy_2022_2027.pdf