Language models, such as ChatGPT, Google Bard, claude.ai and pi.ai, are artificial intelligence-powered chatbots that generate intelligent-sounding text when responding to user prompts. Students often encounter them directly or indirectly. How students should employ language model tools has been debated since the public release of ChatGPT in November 2022 (see Zirar, 2023). Some argue that language model tools enhance the student learning experience. Those concerned say that language model tools restrict student learning.

It is tempting to propose that students should only use language model tools as virtual tutors and in the development of early drafts of work, and that they should have the output of these tools thoroughly checked (Farrokhnia et al., 2023). However, students may habitually generate assessed work rather than going through the learning involved (Zirar, 2023). Students who rely heavily on language model tools without verifying the information lose essential skills like critical thinking and analysis (Wu & Yu, 2024).

My recent article in Review of Education (see Zirar, 2023) provides a synthesis based on a review of 25 academic articles, which highlighted two principal themes:

1. Students’ exposure to language model tools is only helpful with increased awareness of the limitations of such tools. Student learning can focus on ‘original thoughts’, learning by doing, creative use of knowledge in new settings, and editing and fact-checking.
2. Due to the nature of the output of language model tools, human educators will find the output as a source of inspiration or suggestion rather than factual, valid and reliable.

Accordingly, it is reasonable to argue that language model tools can ‘play a specific and defined role’ rather than the whole role in student learning and assessment. Reliance on them without critical evaluation negatively impacts student learning. On the other hand, human educators must actively review and edit the teaching and assessment material generated by language models and any personalised learning strategies, teaching interventions and resources recommended by such tools (Zirar, 2023).

‘Reliance on language models without critical evaluation negatively impacts student learning.’

In summary, language models enhance the learning experience but pose potential risks to student learning if used uncritically. The use of language model tools in education is a double-edged sword. On the one hand, they can be virtual tutors. On the other hand, over-reliance on these tools without critical evaluation can negatively impact student learning, leading to a loss of essential skills like critical thinking and analysis. Academic institutions must increase student awareness of the limitations of such tools. Students should be encouraged to focus on ‘original thoughts’, learning by doing, creatively using knowledge in new settings, and editing and fact-checking. So, how can we effectively integrate language model tools into the learning process while ensuring students maintain and develop essential skills? How can we balance leveraging the benefits of these tools and mitigating their potential drawbacks? These are the questions that educators, students and AI developers must consider as we navigate the future of education in the AI era.

This blog post is based on the article ‘Exploring the impact of language models, such as ChatGPT, on student learning and assessment’ by Araz Zirar, published in the Review of Education.

References

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International. Advance online publication. https://doi.org/10.1080/14703297.2023.2195846

Wu, R., & Yu, Z. (2024). Do AI chatbots improve students learning outcomes? Evidence from a meta-analysis. British Journal of Educational Technology, 55(1), 10–33. https://doi.org/10.1111/bjet.13334  

Zirar, A. (2023). Exploring the impact of language models, such as ChatGPT, on student learning and assessment. Review of Education, 11(3), e3433. https://doi.org/10.1002/rev3.3433