Achievement

Videos are often accompanied by narration delivered either by an audio stream or by captions. Eye movements were recorded while viewing videos with audio narration, captions, or both. A surprisingly large proportion of time was spent reading captions even in the presence of redundant audio. Redundant audio did not speed up reading, but did lead to skipping some portions of the captions. In the absence of captions, eye fixations were drawn to regions with a high density of information, such as the center or regions with high levels of motion. The strong attraction to captions, which initially appears suboptimal, may be based on learned attraction of the line of sight to text; moment by moment evaluations of the relative importance of text vs. video; or the drive to integrate visual and audio streams into a single narrative.

Ross and Kowler (2013). Eye movements while viewing narrated, captioned, and silent videos. Journal of Vision, 13(4):1, 1-19.