1887
Volume 13, Issue 1
  • ISSN 1879-7865
  • E-ISSN: 1879-7873
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This article provides an overview of research showing the advantages of audiovisual input in enhancing second/foreign language learning. It reviews the theoretical frameworks that explain the processing of audiovisual input and its language learning outcomes, presents the multimodal complexity of audiovisual input, and discusses its various components, focusing largely on research findings that indicate the potential of subtitles and captions for enhancing language learning. The paper goes on to address the learning that occurs through audiovisual input and issues related to processing and attention, examined primarily through eye-tracking data. Lastly, it discusses learner-specific factors, namely the role of working memory and aptitude, and learner perceptions of the viewing experience.

Loading

Article metrics loading...

/content/journals/10.1075/lia.22001.mun
2022-08-24
2025-04-22
Loading full text...

Full text loading...

References

  1. Ayres, P., & Sweller, J.
    (2014) The split-attention principle in multimedia learning. InR. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp.206–226). Cambridge University Press. 10.1017/CBO9781139547369.011
    https://doi.org/10.1017/CBO9781139547369.011 [Google Scholar]
  2. Baddeley, A.
    (2012) Working memory: Theories, models and controversies. Annual Review of Psychology, 63, 1–29. 10.1146/annurev‑psych‑120710‑100422
    https://doi.org/10.1146/annurev-psych-120710-100422 [Google Scholar]
  3. Barón, J., & Muñoz, C.
    (in press). Exploring the effects of audiovisual material and age in pragmatic learning. Revista Española de Lingüística Aplicada.
    [Google Scholar]
  4. Benson, P.
    (2011) Language learning and teaching beyond the classroom: An introduction to the field. InP. Benson, & H. Reinders (Eds.), Beyond the language classroom (pp.7–16). Palgrave Macmillan. 10.1057/9780230306790_2
    https://doi.org/10.1057/9780230306790_2 [Google Scholar]
  5. Bianchi, F., & Ciabattoni, T.
    (2008) Captions and subtitles in EFL learning: An investigative study in a comprehensive computer environment. InA. Baldry, M. Pavesi, & C. Taylor Torsello, (Eds.), From didactas to Ecolingua (pp.69–80). Edizioni Universita di Trieste.
    [Google Scholar]
  6. Birulés-Muntané, J., & Soto-Faraco, S.
    (2016) Watching subtitled films can help learning foreign languages. PloS One, 11(6).e0158409. 10.1371/journal.pone.0158409
    https://doi.org/10.1371/journal.pone.0158409 [Google Scholar]
  7. Bisson, M. J., Van Heuven, W. J., Conklin, K., & Tunney, R. J.
    (2012) Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholinguistics, 35(2), 399–418. 10.1017/S0142716412000434
    https://doi.org/10.1017/S0142716412000434 [Google Scholar]
  8. Chai, J., & Erlam, R.
    (2008) The effect and the influence of the use of video and captions on second language learning. New Zealand Studies in Applied Linguistics, 14, 25–44.
    [Google Scholar]
  9. Charles, T. J., & Trenkic, D.
    (2015) Speech segmentation in a second language: The role of bimodal input. InY. Gambier, A. Caimi, & C. Mariotti (Eds.), Subtitles and language learning: Principles, strategies and practical experiences (pp.173–198). Peter Lang.
    [Google Scholar]
  10. Cintrón-Valentín, M., García-Amaya, L., & Ellis, N. C.
    (2019) Captioning and grammar learning in the L2 Spanish classroom. The Language Learning Journal, 47(4), 1–21. 10.1080/09571736.2019.1615978
    https://doi.org/10.1080/09571736.2019.1615978 [Google Scholar]
  11. Collins, L., & Muñoz, C.
    (2016) The foreign language classroom: Current perspectives and future considerations. The Modern Language Journal, 100, 133–147. 10.1111/modl.12305
    https://doi.org/10.1111/modl.12305 [Google Scholar]
  12. Danan, M.
    (2004) Captioning and subtitling: Undervalued language learning strategies. Meta: Journal des traducteurs/Meta: Translators’ Journal, 49(1), 67–77. 10.7202/009021ar
    https://doi.org/10.7202/009021ar [Google Scholar]
  13. Dressman, M.
    (2020) Multimodality and language learning. InM. Dressman, & R. W. Sadler (Eds.), The handbook of informal language learning (pp.39–55). Wiley Blackwell.
    [Google Scholar]
  14. Duchowski, A. T.
    (2002) A breadth-first survey of eye tracking applications. Behavior Research Methods, Instruments, & Computers, 34, 455–470. 10.3758/BF03195475
    https://doi.org/10.3758/BF03195475 [Google Scholar]
  15. Erlam, R.
    (2005) Language aptitude and its relationship to instructional effectiveness in second language acquisition. Language Teaching Research, 9, 147–171. 10.1191/1362168805lr161oa
    https://doi.org/10.1191/1362168805lr161oa [Google Scholar]
  16. Finger-Bou, R.
    (2021) The effects of regular and enhanced captions on incidental vocabulary acquisition. [Unpublished Master’s Dissertation, University of Barcelona].
    [Google Scholar]
  17. Gass, S., Winke, P., Isbell, D. R., & Ahn, J.
    (2019) How captions help people learn languages. A working-memory, eye-tracking study. Language Learning & Technology, 23(2), 84–104.
    [Google Scholar]
  18. Gesa, F.
    (2019) L1 / L2 subtitled TV series and EFL learning: A study on vocabulary acquisition and content comprehension at different proficiency levels. [Doctoral dissertation, Universitat de Barcelona]. UB Campus Repository. hdl.handle.net/2445/149505
    [Google Scholar]
  19. Godfroid, A., Boers, F., & Housen, A.
    (2013) An eye for words: Gauging the role of attention in incidental L2 vocabulary acquisition by means of eye-tracking. Studies in Second Language Acquisition, 35(3), 483–517. https://www.jstor.org/stable/26336219. 10.1017/S0272263113000119
    https://doi.org/10.1017/S0272263113000119 [Google Scholar]
  20. Hubbard, P.
    (2020) Leveraging technology to integrate informal language learning within classroom settings. InM. Dressman, & R. W. Sadler (Eds.), The handbook of informal language learning (pp.405–419). Wiley Blackwell.
    [Google Scholar]
  21. Hulstijn, J. H.
    (2003) Incidental and intentional learning. InC. J. Doughty, & M. H. Long (Eds.), The handbook of second language acquisition (pp.349–381). Blackwell. 10.1002/9780470756492.ch12
    https://doi.org/10.1002/9780470756492.ch12 [Google Scholar]
  22. Jewitt, C.
    (2013) Multimodal methods for researching digital technologies. InS. Price, C. Jewitt, & B. Brown (Eds.), The SAGE handbook of digital technology research (pp.250–265). Sage Publications Limited. 10.4135/9781446282229.n18
    https://doi.org/10.4135/9781446282229.n18 [Google Scholar]
  23. Jordan, G., & Long, M.
    (in press). English language teaching: The way it is now, and the way it could be. Cambridge Scholars.
    [Google Scholar]
  24. Kam, E. F., Liu, Y.-T., & Tseng, W.-T.
    (2020) Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes. ReCALL, 32(2): 213–230. 10.1017/S0958344020000014
    https://doi.org/10.1017/S0958344020000014 [Google Scholar]
  25. Krashen, S. D.
    (1985) The input hypothesis: Issues and implications. Longman.
    [Google Scholar]
  26. Kruger, J-L., Hefer-Jordaan, E., & Matthew, G.
    (2013) Measuring the impact of subtitles on cognitive load: Eye tracking and dynamic audiovisual texts. InProceedings of the 2013 Conference on Eye Tracking South Africa (pp.62–66). Association for Computing Machinery. 10.1145/2509315.2509331
    https://doi.org/10.1145/2509315.2509331 [Google Scholar]
  27. Kruger, J. L., & Steyn, F.
    (2014) Subtitles and eye tracking: Reading and performance. Reading Research Quarterly, 49(1), 105–120. 10.1002/rrq.59
    https://doi.org/10.1002/rrq.59 [Google Scholar]
  28. Kusyk, M., & Sockett, G.
    (2012) From informal resource usage to incidental language acquisition: Language uptake from online television viewing in English. Asp, la revue dun GERAS, 62, 45–65. 10.4000/asp.3104
    https://doi.org/10.4000/asp.3104 [Google Scholar]
  29. Lee, M., & Révész, A.
    (2020) Promoting grammatical development through captions and textual enhancement in multimodal input-based tasks. Studies in Second Language Acquisition, 42 (3), 625–651. 10.1017/S0272263120000108
    https://doi.org/10.1017/S0272263120000108 [Google Scholar]
  30. Leow, R., & Martin, A.
    (2017) Enhancing the input to promote salience of the L2. InS. M. Gass, P. Spinner, & J. Behney (Eds.), Salience in Second Language Acquisition (pp.167–186). Routledge. 10.4324/9781315399027‑9
    https://doi.org/10.4324/9781315399027-9 [Google Scholar]
  31. Long, M. H.
    (2020) Optimal input for language learning: Genuine, simplified, elaborated or modified elaborated?Language Teaching, 53(2): 169–182. 10.1017/S0261444819000466
    https://doi.org/10.1017/S0261444819000466 [Google Scholar]
  32. Majuddin, E.
    (2020) Incidental and intentional acquisition of multiword expressions from audio-visual input: The effects of typographically enhanced captions and repetition [Doctoral dissertation, Victoria University of Wellington].
    [Google Scholar]
  33. Mariotti, C.
    (2015) A survey on stakeholders’ perceptions of subtitles as a means to promote foreign language learning. InY. Gambier, A. Caimi, & C. Mariotti (Eds.), Subtitles and language learning. Principles, strategies and practical experiences (pp.83–102). Peter Lang.
    [Google Scholar]
  34. Mayer, R. E.
    (2009) Multimedia learning. Cambridge University Press. 10.1017/CBO9780511811678
    https://doi.org/10.1017/CBO9780511811678 [Google Scholar]
  35. Mayer, R. E., Lee, H., & Peebles, A.
    (2014) Multimedia learning in a second language: A cognitive load perspective. Applied Cognitive Psychology, 28(5), 653–660. 10.1002/acp.3050
    https://doi.org/10.1002/acp.3050 [Google Scholar]
  36. Mayer, R. E., & Moreno, R.
    (2003) Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. 10.1207/S15326985EP3801_6
    https://doi.org/10.1207/S15326985EP3801_6 [Google Scholar]
  37. Mitterer, H., & McQueen, J.
    (2009) Foreign subtitles help but native-language subtitles harm foreign speech perception. PloS One, 4(11): e7785. 10.1371/journal.pone.0007785
    https://doi.org/10.1371/journal.pone.0007785 [Google Scholar]
  38. Montero Perez, M.
    (2020) Incidental vocabulary learning through viewing video. The role of vocabulary knowledge and working memory. Studies in Second Language Acquisition, 42(4), 749–773. 10.1017/S0272263119000706
    https://doi.org/10.1017/S0272263119000706 [Google Scholar]
  39. Montero Perez, M., Van Den Noortgate, W., & Desmet, P.
    (2013) Captioned video for L2 listening and vocabulary learning. A meta-analysis. System, 41(3), 720–739. 10.1016/j.system.2013.07.013
    https://doi.org/10.1016/j.system.2013.07.013 [Google Scholar]
  40. Montero Perez, M., Peters, E., Clarebout, G., & Desmet, P.
    (2014) Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning & Technology, 18(1), 118–141.
    [Google Scholar]
  41. Montero Perez, M., Peters, E., & Desmet, P.
    (2015) Enhancing vocabulary learning through captioned video: An eye-tracking study. The Modern Language Journal, 99(2), 308–328. 10.1111/modl.12215
    https://doi.org/10.1111/modl.12215 [Google Scholar]
  42. (2018) Vocabulary learning through viewing video: The effect of two enhancement techniques. Computer Assisted Language Learning, 31, 1–26. 10.1080/09588221.2017.1375960
    https://doi.org/10.1080/09588221.2017.1375960 [Google Scholar]
  43. Muñoz, C.
    (2008) Symmetries and asymmetries of age effects in naturalistic and instructed L2 learning. Applied Linguistics, 24(4), 578–596. 10.1093/applin/amm056
    https://doi.org/10.1093/applin/amm056 [Google Scholar]
  44. (2017) The role of age and proficiency in subtitle reading. An eye-tracking study. System, 67, 77–86. 10.1016/j.system.2017.04.015
    https://doi.org/10.1016/j.system.2017.04.015 [Google Scholar]
  45. Paivio, A.
    (1991) Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue canadienne de psychologie, 45(3), 255–87. 10.1037/h0084295
    https://doi.org/10.1037/h0084295 [Google Scholar]
  46. Pattemore, A., & Muñoz, C.
    (2020) Learning L2 constructions from captioned audio-visual exposure: The effect of learner-related factors. System, 93, 1–13. 10.1016/j.system.2020.102303
    https://doi.org/10.1016/j.system.2020.102303 [Google Scholar]
  47. Pattemore, A., Suárez, M. M., & Muñoz, C.
    (2020) Exploring L2 TV preferences and perceptions: Feeling of learning and viewing mode. Short Papers. Eurocall 2020.
    [Google Scholar]
  48. (2021, July1–3). Learning L2 grammar constructions through audio-visual input: The role of aptitude. [Paper presentation] 30th EuroSLA Conference, University of Barcelona, Spain.
    [Google Scholar]
  49. Pavesi, M., & Ghia, E.
    (2020) Informal contact with English. A case study of Italian postgraduate students. Edizioni ETS.
    [Google Scholar]
  50. Pellicer-Sánchez, A., & Boers, F.
    (2019) Pedagogical approaches to the teaching and learning of formulaic language. InA. Siyanova-Chanturia, & A. Pellicer-Sánchez (Eds.), Understanding formulaic language: A second language acquisition perspective (pp.153–173). Routledge.
    [Google Scholar]
  51. Peters, E.
    (2019) The effect of imagery and on-screen text on foreign language vocabulary learning from audiovisual input. TESOL Quarterly, 53(4), 1008–1032. 10.1002/tesq.531
    https://doi.org/10.1002/tesq.531 [Google Scholar]
  52. Peters, E., & Webb, S.
    (2018) Incidental vocabulary acquisition through viewing L2 television and factors that affect learning. Studies in Second Language Acquisition, 40, 551–577. 10.1017/S0272263117000407
    https://doi.org/10.1017/S0272263117000407 [Google Scholar]
  53. Peters, E., Heynen, E., & Puimege, E.
    (2016) Learning vocabulary through audiovisual input: The differential effect of L1 subtitles and captions. System, 63, 134–148. 10.1016/j.system.2016.10.002
    https://doi.org/10.1016/j.system.2016.10.002 [Google Scholar]
  54. Prensky, M.
    (2001) Digital natives, digital immigrants. On the horizon, 9(5), 1–6. 10.1108/10748120110424816
    https://doi.org/10.1108/10748120110424816 [Google Scholar]
  55. Pujadas, G.
    (2019) Language learning through extensive TV viewing. A study with adolescent EFL learners [Doctoral dissertation, University of Barcelona]. UB Campus Repository. diposit.ub.edu/dspace/handle/2445/146118
    [Google Scholar]
  56. Pujadas, G., & Muñoz, C.
    (2017, April19–21). Learning through subtitles. Learners’ preferences and task perception. [Paper presentation] International Conference on Task-Based Language Teaching, Barcelona, Spain.
    [Google Scholar]
  57. (2018, October26–28). What words do we learn better through TV series? The effects of age, proficiency and type of instruction. [Paper presentation] Second Language Research Forum, Montreal, Canada.
    [Google Scholar]
  58. (2019) Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents, The Language Learning Journal, 47(4), 479–496. 10.1080/09571736.2019.1616806
    https://doi.org/10.1080/09571736.2019.1616806 [Google Scholar]
  59. (2020) Examining adolescent EFL learners’ TV viewing comprehension through captions and subtitles. Studies in Second Language Acquisition, 42(3), 551–575. 10.1017/S0272263120000042
    https://doi.org/10.1017/S0272263120000042 [Google Scholar]
  60. (2021, July1–3). Repeating or seeing? Frequency and imagery in vocabulary learning through TV viewing. [Paper presentation] 30th EuroSLA Conference, University of Barcelona., Spain.
    [Google Scholar]
  61. Rayner, K., Slattery, T. J., & Bélanger, N. N.
    (2010) Eye movements, the perceptual span, and reading speed. Psychonomic Bulletin & Review, 17(6), 834-e839. 10.3758/PBR.17.6.834
    https://doi.org/10.3758/PBR.17.6.834 [Google Scholar]
  62. Rodgers, M. P. H.
    (2013) English language learning through viewing television: An investigation of comprehension, incidental vocabulary acquisition, lexical coverage, attitudes and captions [Doctoral dissertation, Victoria University of Wellington].
    [Google Scholar]
  63. (2018) The images in television programs and the potential for learning unknown words. ITL – International Journal of Applied Linguistics, 169(1), 191–211. 10.1075/itl.00012.rod
    https://doi.org/10.1075/itl.00012.rod [Google Scholar]
  64. Rodgers, M. P. H., & Webb, S.
    (2020) Incidental vocabulary learning through viewing television. ITL – International Journal of Applied Linguistics, 171(2), 191–220. 10.1075/itl.18034.rod
    https://doi.org/10.1075/itl.18034.rod [Google Scholar]
  65. Sergeeva, M.
    (2021) Subtitled audiovisual materials in the FL classroom: Teachers’ perspective. [Master’s Thesis, University of Barcelona].
    [Google Scholar]
  66. Suárez, M., & Gesa, F.
    (2019) Learning vocabulary with the support of sustained exposure to captioned video: Do proficiency and aptitude make a difference?The Language Learning Journal, 47(4), 497–517. 10.1080/09571736.2019.1617768
    https://doi.org/10.1080/09571736.2019.1617768 [Google Scholar]
  67. Suzuki, Y.
    (2021) Probing the construct validity of LLAMA_D as a measure of implicit learning aptitude. Incidental instructions, confidence ratings, and reaction time. Studies in Second Language Acquisition, 43(3), 663–676. 10.1017/S0272263120000704
    https://doi.org/10.1017/S0272263120000704 [Google Scholar]
  68. Sweller, J.
    (1988) Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. 10.1207/s15516709cog1202_4
    https://doi.org/10.1207/s15516709cog1202_4 [Google Scholar]
  69. (1999) Instructional design in technical areas. ACER Press.
    [Google Scholar]
  70. Sydorenko, T.
    (2010) Modality of input and vocabulary acquisition. Language Learning & Technology, 14(2), 50–73.
    [Google Scholar]
  71. Uchihara, T., Webb, S., & Yanagisawa, A.
    (2019) The effects of repetition on incidental vocabulary learning: A meta-analysis of correlational studies. Language Learning, 69(3), 559–599. 10.1111/lang.12343
    https://doi.org/10.1111/lang.12343 [Google Scholar]
  72. Vandergrift, L.
    (2007) Recent developments in second language and foreign language listening comprehension research. Language Teaching, 40(3), 191–210. 10.1017/S0261444807004338
    https://doi.org/10.1017/S0261444807004338 [Google Scholar]
  73. Vanderplank, R.
    (1988) The value of teletext sub-titles in language learning. ELT Journal, 42(4), 272–281. 10.1093/elt/42.4.272
    https://doi.org/10.1093/elt/42.4.272 [Google Scholar]
  74. (2016) Captioned media in foreign language learning and teaching: Subtitles for the deaf and hard-of-hearing as tools for language learning. Palgrave Macmillan. 10.1057/978‑1‑137‑50045‑8
    https://doi.org/10.1057/978-1-137-50045-8 [Google Scholar]
  75. Van Lommel, S., Laenen, A., & D’Ydewalle, G.
    (2006) Foreign-grammar acquisition while watching subtitled television programmes. British Journal of Educational Psychology, 76, 243–258. 10.1348/000709905X38946
    https://doi.org/10.1348/000709905X38946 [Google Scholar]
  76. Webb, S., & Rodgers, M. P. H.
    (2009) Vocabulary demands of television programs. Language Learning, 59(2), 335–366. 10.1111/j.1467‑9922.2009.00509.x
    https://doi.org/10.1111/j.1467-9922.2009.00509.x [Google Scholar]
  77. Wiley, J., Sanchez, C. A., & Jaeger, A. J.
    (2014) The individual differences in working memory capacity principle in multimedia learning. InR. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed.) (pp.598–619). Cambridge University Press. 10.1017/CBO9781139547369.029
    https://doi.org/10.1017/CBO9781139547369.029 [Google Scholar]
  78. Winke, P., Gass, S., & Syodorenko, T.
    (2010) The effects of captioning videos used for foreign language listening activities. Language Learning & Technology, 14(1), 65–86.
    [Google Scholar]
  79. Winke, P., Sydorenko, T., & Gass, S.
    (2013) Factors influencing the use of captions by foreign language learners, an eye-tracking study. The Modern Language Journal, 97, 254–275. 10.1111/j.1540‑4781.2013.01432.x
    https://doi.org/10.1111/j.1540-4781.2013.01432.x [Google Scholar]
  80. Wisniewska, N., & Mora, J. C.
    (2020) Can captioned video benefit second language pronunciation?Studies in Second Language Acquisition, 42(3), 599–624. 10.1017/S0272263120000029
    https://doi.org/10.1017/S0272263120000029 [Google Scholar]
/content/journals/10.1075/lia.22001.mun
Loading
  • Article Type: Research Article
Keyword(s): captions; cognitive factors; learner perceptions; multimodal input; subtitles

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error