With Covid-19 pushing video deeper into our pandemic-affected classrooms, studying multimedia design has taken on a new urgency. My recent article in the British Journal of Educational Technology (Garrett, 2020) measures the effect of adding system-controlled pauses to a video. This ‘segmentation’ technique breaks a video into sections, with the goal of not overloading students.

My project used segmentation to improve a Microsoft Excel conditional formatting video. Using videos to teach software has shown inconsistent outcomes (van der Meij & van der Meij, 2014). As instruction moves out of a lab to an online format, understanding why videos work (or not) becomes more critical.

‘As instruction moves out of a lab to an online format, understanding why videos work (or not) becomes more critical.’

The experiment compared three different teaching methods: text with accompanying screenshots, normal video and an auto-paused video. Each method had identical content, giving a high-level overview of the task, a step-by-step tutorial and an ‘on your own’ application task. The experiment was run in two phases over several years, with a total of 71 successful performance measurements (and between 9 and 19 participants per condition). The segmented video automatically stopped at the end of each step, showing ‘continue’ and ‘replay’ buttons, as well as a ‘scrubber’ bar on the bottom (see figure 1).

Figure 1: Segmented video screen

Oddly, the results showed that segmentation was not associated with either a reduction in student effort or an improvement in performance. Student effort was measured with a validated cognitive load scale (Leppink, Paas, van Gog, van der Vleuten, & van Merriënboer, 2014). Student performance was measured by time required to complete the ‘on your own’ application task.

One possible explanation was that the task was not sufficiently challenging for the students. So in response, I increased the video speed by 40 per cent, and re-ran the experiment as Phase 2. The increased speed made it almost impossible for viewers to stay with the step-by-step process. I hypothesised that the increased stress on users would show segmentation’s benefit.

But again, the experiment showed opposite results (see table 1). The increased speed of the video meant that students using the non-segmented video were forced to use the pause/play control, as well as use the scrubber to replay parts of the video. However, they were still more successful with this method than they were with the segmented video, which automatically paused after each step. Students were still able to be successful with the experiment, not even complaining about the speed of the task. Instead, any frustrations expressed during the experiment tended to be inwardly focused, as they blamed themselves for not being able to stay with the fast-paced video.

Table 1: Success measures by experimental condition

The results cast doubt on the benefits of segmentation for software instruction. Students showed more skill with navigating videos than I had expected, even in the presence of a fast-talking instructor. Additionally, this project shows the importance of a realistic baseline condition. Many of the previous studies on segmentation compared system-controlled pauses against videos with no (or very limited) user control (Rey et al., 2019).

Overall, this study shows that adding system-controlled pauses to software videos is a poor use of faculty time. Instead, instructors should focus on ensuring that videos are easy to navigate.

This blog is based on the article ‘Segmentation’s failure to improve software video tutorials’ by Nathan Garrett, published in the British Journal of Educational Technology. It has been made free-to-view for those without a subscription for a limited period, courtesy of our publisher, Wiley.

References

Garrett, N. (2020). Segmentation’s failure to improve software video tutorials. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13000

Leppink, J., Paas, F., van Gog, T., van der Vleuten, C. P. M., & van Merriënboer, J. J. G. (2014). Effects of pairs of problems and examples on task performance and different types of cognitive load. Learning and Instruction, 30, 32–42. https://doi.org/10.1016/j.learninstruc.2013.12.001

van der Meij, J., & van der Meij, H. (2015). A test of the design of a video tutorial for software training. Journal of Computer Assisted Learning, 31, 116–132. https://doi.org/10.1111/jcal.12082

Rey, G. D., Beege, M., Nebel, S., Wirzberger, M., Schmitt, T. H., & Schneider, S. (2019). A Meta-analysis of the segmenting effect. In Educational Psychology Review, 31, 389–419. https://doi.org/10.1007/s10648-018-9456-4