Immersive virtual reality (VR) is revolutionising education by offering highly personalised learning experiences. To fully leverage this transformative potential, it is essential to deepen our understanding of the self-regulatory processes that underpin VR learning (). Self-regulated learning (SRL) is critical for successful learning: it involves monitoring and controlling one’s behavior, emotions and thoughts, and adapting them to different situations according to one’s goals (Winne & Hadwin, 1998). SRL is particularly important in VR environments, as they are often more complex, dynamic and immersive than traditional learning settings, requiring learners to actively oversee their progress and make strategic adjustments as needed (Azevedo & Gašević, 2019). However, identifying the need for self-regulation and effectively regulating one’s learning can be a challenge (Winne & Azevedo, 2022). It is crucial, therefore, to detect the moments that the need for regulation is recognised through metacognitive monitoring, the core component of the SRL process.

Our recent study set out to pinpoint these pivotal moments using multimodal data collected during VR learning (Sobocinski et al., 2023). In immersive VR environments, learners physically engage with virtual learning materials and tasks in a unique interaction that provides insights into self-regulation not easily observed elsewhere. While the concept of embodied cognition has explored how bodily movement influences cognition (Johnson-Glenberg, 2018), there has been limited research into the role of movement in self-regulated learning within VR environments.

We harnessed various data sources, including bird’s-eye-view videos to track movement, screen recordings for capturing learners’ actions in the virtual environment, physiological metrics such as heart rate variability to indicate cognitive load, and learners’ verbalisations through think-aloud protocols to capture metacognitive monitoring.

Our holistic approach aimed to decode the intricate processes inherent in SRL, specifically focusing on metacognitive monitoring and control within genuine learning contexts, a focus underscored in recent literature (Azevedo & Gašević, 2019; Molenaar et al., 2023). Our study centres on capturing and measuring SRL processes, examining how metacognitive monitoring (via think-aloud data) responds to cognitive load (monitored through heart rate variability) and how this interaction triggers behavioural changes, reflected in learners’ physical movements. For example, the cognitive load measure (heart rate variability) would respond to a learner struggling with defining an equation, followed by a learner voicing out loud ‘this is difficult’ (verbalised metacognitive monitoring), which would be followed by them moving in the virtual environment looking at simulations or further instructions that could help them solve the problem (movement).

‘Our study centres on capturing and measuring self-regulated learning processes, examining how metacognitive monitoring responds to cognitive load and how this interaction triggers behavioural changes, reflected in learners’ physical movements.’

In conclusion, our research ventures into the realm of sensor data and advanced analytics, shedding light on SRL processes in immersive VR learning, in line with the latest trends in educational research of moving towards new applications to support learners to successfully regulate their learning in advanced learning technologies (Molenaar et al., 2023) . By revealing the connections in VR-based learning between cognitive load, metacognitive monitoring and motion, our study contributes to a deeper understanding of self-regulated learning processes within VR environments.

Our findings hold promise for future research. While there is ample exploration of adaptive VR environments in various fields, there is a dearth of studies investigating how VR environments can adapt to learners’ unique self-regulatory processes, as captured through multimodal data. By integrating such data, we can offer real-time, adaptive and personalised support in VR learning environments, ensuring each learner receives tailored assistance, paving the way for more seamless and effective learning experiences in authentic settings.

This blog post is based on the article ‘Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal data’ by Marta Sobocinski, Daryn Dever, Megan Wiedbusch, Foysal Mubarak, Roger Azevedo and Sanna Järvelä, published in the British Journal of Educational Technology.

References

Azevedo, R., & Gašević, D. (2019). Analyzing multimodal multichannel data about self-regulated learning with advanced learning technologies: Issues and challenges. Computers in Human Behavior, 96, 207–210. https://doi.org/10.1016/j.chb.2019.03.025  

Johnson-Glenberg, M. C. (2018). Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers Robotics AI, 5, 1–19. https://doi.org/10.3389/frobt.2018.00081

Lui, L. F., Radhakrishnan, U., Chinello, F., & Koumaditis, K. (2023). Adaptive Immersive VR Training Based on Performance and Self-Efficacy. 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 25–29. https://doi.org/10.1109/VRW58643.2023.00012

Molenaar, I., Mooij, S. de, Azevedo, R., Bannert, M., Järvelä, S., & Gašević, D. (2023). Measuring self-regulated learning and the role of AI: Five years of research using multimodal multichannel data. Computers in Human Behavior, 139, 107540. https://doi.org/10.1016/j.chb.2022.107540

Sobocinski, M., Dever, D., Wiedbusch, M., Mubarak, F., Azevedo, R., & Järvelä, S. (2023). Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal data. British Journal of Educational Technology. Advance online publication. https://doi.org/10.1111/bjet.13393

Winne, P. H., & Azevedo, R. (2022). Metacognition and self-regulated learning. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (3rd ed., pp. 93–113). Cambridge University Press. https://doi.org/10.1017/9781108888295.007

Winne, P. H., & Hadwin, A. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277–304). Routledge.