Volume 28, Issue 2
  • ISSN 0924-1884
  • E-ISSN: 1569-9986
Buy:$35.00 + Taxes


Psycholinguistic investigations of translated audiovisual products have been conducted since at least the 1980s. These mainly concerned the role of subtitles in the processing of language in the context of language acquisition, literacy, and education. This article provides an overview of some of the most productive lines of research from a psycholinguistic angle in audiovisual translation (AVT), focussing on studies that investigated the positive effects of subtitles on language performance, but also on a growing body of behavioural research on the cognitive processing of the language of subtitles. The article evaluates a number of methodologies in some of the most prominent studies on the processing of subtitles, primarily making use of eye tracking, and then provides some thoughts on future directions in psycholinguistic studies on the processing of the language of AVT.


Article metrics loading...

Loading full text...

Full text loading...


  1. Bird, Stephen A. , and John Williams
    2002 “The Effect of Bimodal Input on Implicit and Explicit Memory: An Investigation into the Benefits of Within-Language Subtitling.” Applied Psycholinguistics23 (4): 509–533. doi: 10.1017/S0142716402004022
    https://doi.org/10.1017/S0142716402004022 [Google Scholar]
  2. Bisson, Marie-Josée , Walter Van Heuven , Kathy Conklin , and Richard Tunney
    2012 “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics35: 399–418. doi: 10.1017/S0142716412000434
    https://doi.org/10.1017/S0142716412000434 [Google Scholar]
  3. Danan, Martine
    2004 “Captioning and Subtitling: Undervalued Language Learning Strategies.” Meta49 (1): 67–77. doi: 10.7202/009021ar
    https://doi.org/10.7202/009021ar [Google Scholar]
  4. Diao, Yali , Paul Chandler , and John Sweller
    2007 “The Effect of Written Text on Comprehension of Spoken English as a Foreign Language.” The American Journal of Psychology120 (2): 237–261.
    [Google Scholar]
  5. d’Ydewalle, Géry , and Wim De Bruycker
    2007 “Eye Movements of Children and Adults While Reading Television Subtitles.” European Psychologist12: 196–205. doi: 10.1027/1016‑9040.12.3.196
    https://doi.org/10.1027/1016-9040.12.3.196 [Google Scholar]
  6. d’Ydewalle, Géry , and Ingrid Gielen
    1992 “Attention Allocation with Overlapping Sound, Image, and Text.” InEye Movements and Visual Cognition: Scene Perception and Reading, ed. by Keith Rayner , 415–427. New York: Springer-Verlag. doi: 10.1007/978‑1‑4612‑2852‑3_25
    https://doi.org/10.1007/978-1-4612-2852-3_25 [Google Scholar]
  7. d’Ydewalle, Géry , Caroline Praet , Karl Verfaillie , and Johan van Rensbergen
    1991 “Watching Subtitled Television: Automatic Reading Behavior.” Communication Research18: 650–665. doi: 10.1177/009365091018005005
    https://doi.org/10.1177/009365091018005005 [Google Scholar]
  8. Garza, Thomas J.
    1991 “Evaluating the Use of Captioned Video Materials in Advanced Foreign Language Learning.” Foreign Language Annuals24 (3): 239–258. doi: 10.1111/j.1944‑9720.1991.tb00469.x
    https://doi.org/10.1111/j.1944-9720.1991.tb00469.x [Google Scholar]
  9. Ghia, Elisa
    2012 “The Impact of Translation Strategies on Subtitle Reading.” InEye Tracking in Audiovisual Translation, ed. by Elisa Perego , 155–182. Roma: Aracne Editrice.
    [Google Scholar]
  10. Just, Marcel Adam , and Patricia A. Carpenter
    1980 “A Theory of Reading: From Eye Fixations to Comprehension.” Psychological Review87 (4): 329–354. doi: 10.1037/0033‑295X.87.4.329
    https://doi.org/10.1037/0033-295X.87.4.329 [Google Scholar]
  11. Keating, Gregory D.
    2014 “Eye Tracking with Text.” InResearch Methods in Second Language Psycholinguistics, ed. by Jill Jegerski , and Bill VanPatten , 69–92. New York: Routledge.
    [Google Scholar]
  12. Krejtz, Izabela , Agnieszka Szarkowska , and Krzysztof Krejtz
    2013 “The Effects of Shot Changes on Eye Movements in Subtitling.” Journal of Eye Movement Research6 (5): 1–12.
    [Google Scholar]
  13. Kruger, Jan-Louis
    2013 “Subtitles in the Classroom: Balancing the Benefits of Dual Coding with the Cost of Increased Cognitive Load.” Journal for Language Teaching47 (1): 29–53.
    [Google Scholar]
  14. Kruger, Jan-Louis , and Faans Steyn
    2014 “Subtitles and Eye Tracking: Reading and Performance.” Reading Research Quarterly49 (1): 105–120.
    [Google Scholar]
  15. Kruger, Jan-Louis , Esté Hefer , and Gordon Matthew
    2013 “Measuring the Impact of Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts.” Proceedings of the 2013 Conference on Eye Tracking South Africa, 62–66. New York: ACM. doi: 10.1145/2509315.2509331
    https://doi.org/10.1145/2509315.2509331 [Google Scholar]
  16. Markham, Peter L.
    1993 “Captioned Television Videotapes: Effects of Visual Support on Second Language Comprehension.” Journal of Educational Technology Systems21: 183–191. doi: 10.2190/59JJ‑0RBN‑KKVL‑C2YG
    https://doi.org/10.2190/59JJ-0RBN-KKVL-C2YG [Google Scholar]
  17. Markham Peter, L.
    1999 “Captioned Video-Tapes and Second Language Listening Word Recognition.” Foreign Language Annals32 (3): 321–328. doi: 10.1111/j.1944‑9720.1999.tb01344.x
    https://doi.org/10.1111/j.1944-9720.1999.tb01344.x [Google Scholar]
  18. Perego, Elisa , Fabio Del Missier , Marco Porta , and Mauro Mosconi
    2010 “The Cognitive Effectiveness of Subtitle Processing.” Media Psychology13 (3): 243–272. doi: 10.1080/15213269.2010.502873
    https://doi.org/10.1080/15213269.2010.502873 [Google Scholar]
  19. Pulvermüller, Friedemann
    2009 “Word Processing in the Brain as Revealed by Neuro­physiological Imaging.” InThe Oxford Handbook of Psycholinguistics, ed. by M. Gareth Gaskell , 119–139. Oxford: Oxford University Press.
    [Google Scholar]
  20. Rajendran, Dhevi , Andrew Duchowski , Pilar Orero , Juan Martínez , and Pablo Romero-Fresco
    2013 “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.” Perspectives: Studies in Translatology21 (1): 5–31. doi: 10.1080/0907676X.2012.722651
    https://doi.org/10.1080/0907676X.2012.722651 [Google Scholar]
  21. Rayner, Keith
    1998 “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin124: 372–422. doi: 10.1037/0033‑2909.124.3.372
    https://doi.org/10.1037/0033-2909.124.3.372 [Google Scholar]
  22. Specker, Elizabeth A.
    2008L1/L2 Eye Movement Reading of Closed Captioning: A Multimodal Analysis of Multimodal Use. PhD diss. University of Arizona.
    [Google Scholar]
  23. Vanderplank, Robert
    1988 “The Value of Teletext Sub-titles in Language Learning.” ELT Journal42 (4): 272–281. doi: 10.1093/elt/42.4.272
    https://doi.org/10.1093/elt/42.4.272 [Google Scholar]
  24. 1990 “Paying Attention to the Words: Practical and Theoretical Problems in Watching Television Programmes with Uni-lingual (CEEFAX) Sub-titles.” System18 (2): 221–234. doi: 10.1016/0346‑251X(90)90056‑B
    https://doi.org/10.1016/0346-251X(90)90056-B [Google Scholar]
  25. 2013 “‘Effects of’ and ‘Effects with’ Captions: How Exactly Does Watching a TV Programme with Same-Language Subtitles Make a Difference to Language Learners?” Language Teaching1–16.
    [Google Scholar]
  26. VanPatten, Bill
    2014 “The Psycholinguistics of SLA.” InResearch Methods in Second Language Psycholinguistics (Second Language Acquisition Research Series), ed. by Jill Jegerski , and Bill VanPatten , 1–19. New York: Routledge.
    [Google Scholar]
  27. Winke, Paula , Susan Gass , and Tetyana Syderenko
    2013 “Factors Influencing the Use of Captions by Foreign Language Learners: An Eye Tracking Study.” The Modern Language Journal97 (1): 254–275. doi: 10.1111/j.1540‑4781.2013.01432.x
    https://doi.org/10.1111/j.1540-4781.2013.01432.x [Google Scholar]
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error