Volume 11, Issue 1
GBP
Buy:£15.00 + Taxes

Abstract

Abstract

In this paper, we aim to promote the use of linear mixed models (LMMs) in eye-tracking research on subtitling. Using eye tracking to study viewers’ reading of subtitles often warrants controlling for many confounding variables. However, even assuming that these variables are known to researchers, such control may not be possible or desired. Traditional statistical methods such as -tests or ANOVAs exacerbate the problem due to the use of aggregated data: each participant has one data point per dependent variable. As a solution, we propose the use of LMMs, which are better suited to account for a number of subtitle and participant characteristics, thus explaining more variance. We introduce essential theoretical aspects of LMMs and highlight some of their advantages over traditional statistical methods. To illustrate our point, we compare two analyses of the same dataset: one using a -test; another using LMMs.

Loading

Article metrics loading...

/content/journals/10.1075/ts.21013.sil
2022-06-14
2024-03-29
Loading full text...

Full text loading...

References

  1. Baayen, Harald
    2008aAnalyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge: Cambridge University Press. 10.1017/CBO9780511801686
    https://doi.org/10.1017/CBO9780511801686 [Google Scholar]
  2. 2008bExploratory Data Analysis: An Introduction to R for the Language Sciences. Cambridge: Cambridge University Press.
    [Google Scholar]
  3. Balling, Laura Winther
    2008 “A Brief Introduction to Regression Designs and Mixed-Effects Modelling by a Recent Convert.” Copenhagen Studies in Language, 175–92.
    [Google Scholar]
  4. Balling, Laura Winther, and Kristian Tangsgaard Hvelplund
    2015 “Design and Statistics in Quantitative Translation (Process) Research”. Translation Spaces4 (1): 170–187. 10.1075/ts.4.1.08bal
    https://doi.org/10.1075/ts.4.1.08bal [Google Scholar]
  5. Barr, Dale. J., Roger Levy, Christoph Scheepers, and Harry J. Tily
    2013 “Random Effects Structure for Confirmatory Hypothesis Testing: Keep It Maximal.” J Mem Lang68 (3), 255–278. 10.1016/j.jml.2012.11.001
    https://doi.org/10.1016/j.jml.2012.11.001 [Google Scholar]
  6. Bates, Douglas, Reinhold Kliegl, Shravan Vasishth, and R. Harald Baayen
    2018 “Parsimonius Mixed Models.” arXiv:1506.04967v2 [stat.ME].
    [Google Scholar]
  7. Bisson, Marie-Josée, Walter J. B. Heuven, Kathy Conklin, and Richard J. Tunney
    2014 “The Role of Repeated Exposure to Multimodal Input in Incidental Acquisition of Foreign Language Vocabulary.” Language Learning64 (4): 855–77. 10.1111/lang.12085
    https://doi.org/10.1111/lang.12085 [Google Scholar]
  8. Bolker, Ben
  9. Brown, Violet A.
    2021 An Introduction to Linear Mixed-Effects Modelling in R. Advances in Methods and Practices in Psychological Science4 (1): 1–19. 10.1177/2515245920960351
    https://doi.org/10.1177/2515245920960351 [Google Scholar]
  10. Brysbaert, Marc, and Boris New
    2009 “Moving beyond Kučera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English.” Behavior Research Methods41 (4): 977–90. 10.3758/BRM.41.4.977
    https://doi.org/10.3758/BRM.41.4.977 [Google Scholar]
  11. Burnham, Denis, Kaoru Sekiyama, Gerard Bailly, Pascal Perrier, and Eric Vatikiotis-Bateson
    2012 “Investigating Auditory-Visual Speech Perception Development.” InAudiovisual Speech Processing, edited byGérard Bailly, Pascal Perrier and Eric Vatikiotis-Bateson, 62–75. Cambridge: Cambridge University Press. 10.1017/CBO9780511843891.006
    https://doi.org/10.1017/CBO9780511843891.006 [Google Scholar]
  12. Cambra, Cristina, Olivier Penacchio, Núria Silvestre, and Aurora Leal
    2014 “Visual Attention to Subtitles When Viewing a Cartoon by Deaf and Hearing Children: An Eye-Tracking Pilot Study.” Perspectives22 (4): 607–17. 10.1080/0907676X.2014.923477
    https://doi.org/10.1080/0907676X.2014.923477 [Google Scholar]
  13. Carson, Robyn J., and Christina M. L. Beeson
    2013 “Crossing Language Barriers: Using Crossed Random Effects Modelling in Psycholinguistics Research.” Tutorials in Quantitative Methods for Psychology9 (1): 25–41. 10.20982/tqmp.09.1.p025
    https://doi.org/10.20982/tqmp.09.1.p025 [Google Scholar]
  14. Crossley, Scott A., Tom Cobb, and Danielle S. McNamara
    2013 “Comparing Count-Based and Band-Based Indices of Word Frequency: Implications for Active Vocabulary Research and Pedagogical Applications.” System41 (4): 965–81. 10.1016/j.system.2013.08.002
    https://doi.org/10.1016/j.system.2013.08.002 [Google Scholar]
  15. Cunnings, Ian, and Ian Finlayson
    2015 “Mixed Effects Modeling and Longitudinal Data Analysis.” InAdvancing Quantitative Methods in Second Language Research, edited byLuke Plonsky, 159–81. New York: Routledge. 10.4324/9781315870908‑8
    https://doi.org/10.4324/9781315870908-8 [Google Scholar]
  16. De Bruycker, Wim, and Gery d’Ydewalle
    2003 “Reading Native and Foreign Language Television Subtitles in Children and Adults.” InThe Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, edited byJukka Hyönä, Ralph Radach, and Heiner Deubel, 671–84. Amsterdam: North-Holland. 10.1016/B978‑044451020‑4/50036‑0
    https://doi.org/10.1016/B978-044451020-4/50036-0 [Google Scholar]
  17. Dijkstra, Ton, Koji Miwa, Bianca Brummelhuis, Maya Sappelli, and Harald Baayen
    2010 “How Cross-Language Similarity and Task Demands Affect Cognate Recognition.” Journal of Memory and Language62 (3): 284–301. 10.1016/j.jml.2009.12.003
    https://doi.org/10.1016/j.jml.2009.12.003 [Google Scholar]
  18. Doherty, Stephen, and Jan-Louis Kruger
    2018 “The Development of Eye Tracking in Empirical Research on Subtitling and Captioning.” InSeeing into Screens: Eye Tracking and the Moving Imaging, edited byJodi Sita, Tessa Dwyer, Sean Redmond, and Claire Perkins, 46–64. London: Bloomsbury. 10.5040/9781501329012.0009
    https://doi.org/10.5040/9781501329012.0009 [Google Scholar]
  19. Duyck, Wouter, Eva Van Assche, Denis Drieghe, and Robert J. Hartsuiker
    2007 “Visual Word Recognition by Bilinguals in a Sentence Context: Evidence for Nonselective Lexical Access.” J Exp Psychol Learn Mem Cogn33 (4): 663–79. 10.1037/0278‑7393.33.4.663
    https://doi.org/10.1037/0278-7393.33.4.663 [Google Scholar]
  20. Díaz-Cintas, Jorge and Aline Remael
    (2021) Subtitling. Concepts and Practices. London and New York, Routledge.
    [Google Scholar]
  21. Ellis, Nick C., and Alan Beaton
    1993 “Psycholinguistic Determinants of Foreign Language Vocabulary Learning.” Language Learning43 (4): 559–617. 10.1111/j.1467‑1770.1993.tb00627.x
    https://doi.org/10.1111/j.1467-1770.1993.tb00627.x [Google Scholar]
  22. Field, Andy
    2017Discovering Statistics Using IBM SPSS Statistics. 5th ed.London: SAGE Publications, Inc.
    [Google Scholar]
  23. Garson, G. David
    2020Multilevel modelling: Applications in Stata®, IBM® SPSS®, SAS®, R, & HLMTM. New York: Sage.
    [Google Scholar]
  24. Gerber-Morón, Olivia, and Agnieszka Szarkowska
    2018 “Line Breaks in Subtitling: An Eye Tracking Study on Viewer Preferences.” Journal of Eye Movement Research11 (3): 1–22. 10.16910/jemr.11.3.2
    https://doi.org/10.16910/jemr.11.3.2 [Google Scholar]
  25. Gerber-Morón, Olivia, Agnieszka Szarkowska, and Bencie Woll
    2018 “The Impact of Text Segmentation on Subtitle Reading.” Journal of Eye Movement Research11 (4): 1–18. 10.16910/11.4.2
    https://doi.org/10.16910/11.4.2 [Google Scholar]
  26. Ghia, Elisa
    2012 “The Impact of Translation Strategies on Subtitle Reading.” InEye Tracking in Audiovisual Translation, edited byElisa Perego, 155–82. Roma: Aracne Editrice.
    [Google Scholar]
  27. Gile, Daniel
    2016 “Experimental research.” InResearching Translation and Interpreting, edited byClaudia V. Angelelli and Brian James Baer, 220–228. New York: Routledge.
    [Google Scholar]
  28. Hajduk, Gabriela K.
    2019 “Introduction to Linear Mixed Models.” https://ourcodingclub.github.io/tutorials/mixed-models/#crossed
  29. Harrison, Xavier. A., Lynda Donaldson, Maria E. Correa-Cano, Julian Evans, David N. Fisher, Cecily E. D. Goodwin, Beth. S. Robinson, David J. Hodgson, and Richard Inger
    2018 “A Brief Introduction to Mixed Effects Modelling and Multi-Model Inference in Ecology.” PeerJ61: e4794. 10.7717/peerj.4794
    https://doi.org/10.7717/peerj.4794 [Google Scholar]
  30. Heck, Ronald H., Scott L. Thomas, and Lynn N. Tabata
    2012Multilevel Modelling of Categorical Outcomes Using IBM SPSS. Quantitative Methodology Series. New York / London: Routledge.
    [Google Scholar]
  31. Hedeker, Donald
    2003 “A Mixed-Effects Multinomial Logistic Regression Model.” Statistics in Medicine22 (9): 1433–46. 10.1002/sim.1522
    https://doi.org/10.1002/sim.1522 [Google Scholar]
  32. Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost van de Weijer
    2011Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.
    [Google Scholar]
  33. Inhoff, Albrecht W., and Keith Rayner
    1986 “Parafoveal Word Processing during Eye Fixations in Reading: Effects of Word Frequency.” Perception & Psychophysics40 (6): 431–39. 10.3758/BF03208203
    https://doi.org/10.3758/BF03208203 [Google Scholar]
  34. Jensema, Carl
    1998 “Viewer Reaction to Different Television Captioning Speeds.” American Annals of the Deaf143 (4): 318–24. 10.1353/aad.2012.0073
    https://doi.org/10.1353/aad.2012.0073 [Google Scholar]
  35. Koolstra, Cees M., Tom H. A. Van Der Voort, and Gery d’Ydewalle
    1999 “Lengthening the Presentation Time of Subtitles on Television: Effects on Children’s Reading Time and Recognition.” Communications24 (4): 407–22. 10.1515/comm.1999.24.4.407
    https://doi.org/10.1515/comm.1999.24.4.407 [Google Scholar]
  36. Krejtz, Izabela, Agnieszka Szarkowska, and Maria Łogińska
    2016 “Reading Function and Content Words in Subtitled Videos.” Journal Of Deaf Studies And Deaf Education21 (2): 222–32. 10.1093/deafed/env061
    https://doi.org/10.1093/deafed/env061 [Google Scholar]
  37. Kruger, Jan-Louis
    2018 “Eye tracking in audiovisual translation research”. InThe Routledge Handbook of Audiovisual Translation, edited byL. Pérez-González, 350–366. Routledge. 10.4324/9781315717166‑22
    https://doi.org/10.4324/9781315717166-22 [Google Scholar]
  38. Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew
    2014 “Attention Distribution and Cognitive Load in a Subtitled Academic Lecture: L1 vs. L2.” Journal of Eye Movement Research7 (5): 1–15. 10.16910/jemr.7.5.4
    https://doi.org/10.16910/jemr.7.5.4 [Google Scholar]
  39. Kruger, Jan-Louis, Stephen Doherty, and María T. Soto-Sanfiel
    2017 “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.” Comunicar, 501 (January): 23–32. 10.3916/C50‑2017‑02
    https://doi.org/10.3916/C50-2017-02 [Google Scholar]
  40. Kruger, Jan-Louis, and Stephen Doherty
    2018 “Triangulation of Online and Offline Measures of Processing and Reception in AVT.” InReception Studies and Audiovisual Translation, edited byElena Di Giovanni and Yves Gambier, 91–109. Amsterdam/New York: John Benjamins. 10.1075/btl.141.06kru
    https://doi.org/10.1075/btl.141.06kru [Google Scholar]
  41. Lemhöfer, Kristin, and Ton Dijkstra
    2004 “Recognizing Cognates and Interlingual Homographs: Effects of Code Similarity in Language-Specific and Generalized Lexical Decision.” Memory & Cognition32 (4): 533–50. 10.3758/BF03195845
    https://doi.org/10.3758/BF03195845 [Google Scholar]
  42. Liao, Sixin, Jan-Louis Kruger, and Stephen Doherty
    2020 “The Impact of Monolingual and Bilingual Subtitles on Visual Attention, Cognitive Load, and Comprehension.” Journal of Specialised Translation331: 70–98.
    [Google Scholar]
  43. Liao, Sixin, Lili Yu, Erik D. Reichle, and Jan-Louis Kruger
    2020 “Using Eye Movements to Study the Reading of Subtitles in Video.” Scientific Studies of Reading25 (5): 417–435. 10.1080/10888438.2020.1823986
    https://doi.org/10.1080/10888438.2020.1823986 [Google Scholar]
  44. Łuczak, Krzysztof
    2017 “The Effects of the Language of the Soundtrack on Film Comprehension, Cognitive Load and Subtitle Reading Patterns. An Eye-Tracking Study.” Institite of Applied Linguistics. Warsaw: University of Warsaw.
    [Google Scholar]
  45. Mellinger, Christopher D., and Thomas A. Hanson
    (2017) Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.
    [Google Scholar]
  46. Meteyard, Lotte, and Robert A. I. Davies
    2020 “Best Practice Guidance for Linear Mixed-Effects Models in Psychological Science.” Journal of Memory and Language1121. 10.1016/j.jml.2020.104092
    https://doi.org/10.1016/j.jml.2020.104092 [Google Scholar]
  47. Moran, Siobhan
    2009 “The Effect of Linguistic Variation on Subtitle Reception.” Toronto: York University.
    [Google Scholar]
  48. Muñoz, Carmen
    2017 “The Role of Age and Proficiency in Subtitle Reading. An Eye-Tracking Study.” System671: 77–86. 10.1016/j.system.2017.04.015
    https://doi.org/10.1016/j.system.2017.04.015 [Google Scholar]
  49. Orrego-Carmona, David
    2015 “The Reception of (Non)Professional Subtitling.” Department of English and German Studies. Tarragona: Universitat Rovira i Virgili.
    [Google Scholar]
  50. Peters, Elke
    2019 “Factors Affecting the Learning of Single-Word Items.” InThe Routledge Handbook of Vocabulary Studies, edited byStuart Webb, 125–42. New York: Routledge. 10.4324/9780429291586‑9
    https://doi.org/10.4324/9780429291586-9 [Google Scholar]
  51. Ragni, Valentina
    2020 “More than Meets the Eye: An Eye-Tracking Study of the Effects of Translation on the Processing and Memorisation of Reversed Subtitles.” Journal of Specialised Translation331: 99–128.
    [Google Scholar]
  52. Rajendran, Dhevi J., Andrew T. Duchowski, Pilar Orero, Juan Martínez, and Pablo Romero-Fresco
    2013 “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.” Perspectives21 (1): 5–21. 10.1080/0907676X.2012.722651
    https://doi.org/10.1080/0907676X.2012.722651 [Google Scholar]
  53. Rayner, Keith
    2009 “Eye movements and attention in reading, scene perception, and visual search”. The Quarterly Journal of Experimental Psychology62(8); 1457–1506. 10.1080/17470210902816461
    https://doi.org/10.1080/17470210902816461 [Google Scholar]
  54. Rayner, Keith, and Susan A. Duffy
    1986 “Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity.” Memory & Cognition14 (3): 191–201. 10.3758/BF03197692
    https://doi.org/10.3758/BF03197692 [Google Scholar]
  55. Rayner, Keith, Tessa Warren, Barbara J. Juhasz, and Simon P. Liversedge
    2004 “The Effect of Plausibility on Eye Movements in Reading.” J Exp Psychol Learn Mem Cogn30 (6): 1290–1301. 10.1037/0278‑7393.30.6.1290
    https://doi.org/10.1037/0278-7393.30.6.1290 [Google Scholar]
  56. Rayner, K., Timothy J. Slattery, Denis Drieghe, and Simon P. Liversedge
    2011 “Eye Movements and Word Skipping during Reading: Effects of Word Length and Predictability.” J Exp Psychol Hum Percept Perform37 (2): 514–28. 10.1037/a0020990
    https://doi.org/10.1037/a0020990 [Google Scholar]
  57. Romero-Fresco, Pablo
    2015 “Final Thoughts: Viewing Speed in Subtitling.” InThe Reception of Subtitles for the Deaf and Hard of Hearing in Europe, edited byPablo Romero-Fresco, 335–41. Bern: Peter Lang.
    [Google Scholar]
  58. RStudio Team
    RStudio Team 2021Studio: Integrated Development for R. RStudio. R. Boston. www.rstudio.com/
    [Google Scholar]
  59. Saldanha, Gabriela, and Sharon O’Brien
    2014Research Methodologies in Translation Studies. New York: Routledge. 10.4324/9781315760100
    https://doi.org/10.4324/9781315760100 [Google Scholar]
  60. Schwartz, Ana I., and Judith F. Kroll
    2006 “Bilingual Lexical Activation in Sentence Context.” Journal of Memory and Language55 (2): 197–212. 10.1016/j.jml.2006.03.004
    https://doi.org/10.1016/j.jml.2006.03.004 [Google Scholar]
  61. Singmann, Henrik, and David Kellen
    2019 “An Introduction to Mixed Models for Experimental Psychology.” InNew Methods in Cognitive Psychology, edited byDaniel Spieler and Eric Schumacher, 1st ed., 4–31. Routledge. 10.4324/9780429318405‑2
    https://doi.org/10.4324/9780429318405-2 [Google Scholar]
  62. Silva, Breno B., David Orrego-Carmona, and Agnieszka Szarkowska
    2021a Data for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’ Figshare. Available at: (accessedMarch 2022). 10.6084/m9.figshare.14403389.v1
    https://doi.org/10.6084/m9.figshare.14403389.v1 [Google Scholar]
  63. 2021b Appendices for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’ Figshare. Available at: (accessedMarch 2022). 10.1075/ts.21013.sil
    https://doi.org/10.1075/ts.21013.sil [Google Scholar]
  64. Szarkowska, Agnieszka, Izabela Krejtz, Zuzanna Kłyszejko, and Anna Wieczorek
    2011 “Verbatim, Standard, or Edited? Reading Patterns of Different Captioning Styles among Deaf, Hard of Hearing, and Hearing Viewers.” American Annals of the Deaf156 (4): 363–78. 10.1353/aad.2011.0039
    https://doi.org/10.1353/aad.2011.0039 [Google Scholar]
  65. Szarkowska, Agnieszka, and Olivia Gerber-Morón
    2018 “Viewers Can Keep up with Fast Subtitles: Evidence from Eye Movements.” Plos One13 (6). 10.1371/journal.pone.0199331
    https://doi.org/10.1371/journal.pone.0199331 [Google Scholar]
  66. 2019 “Two or Three Lines: A Mixed-Methods Study on Subtitle Processing and Preferences.” Perspectives27 (1): 144–64. 10.1080/0907676X.2018.1520267
    https://doi.org/10.1080/0907676X.2018.1520267 [Google Scholar]
  67. Szarkowska, Agnieszka, and Lidia Bogucka
    2019 “Six-Second Rule Revisited: An Eye-Tracking Study on the Impact of Speech Rate and Language Proficiency on Subtitle Reading.” Translation, Cognition & Behavior2 (1): 101–24. 10.1075/tcb.00022.sza
    https://doi.org/10.1075/tcb.00022.sza [Google Scholar]
  68. Szarkowska, Agnieszka, Breno B. Silva, and David Orrego-Carmona
    2021 Effects of subtitle speed on proportional reading time: Re-analysing subtitle reading data with mixed effects models. Translation, Cognition & Behavior, 4(2): 305–330. 10.1075/tcb.00057.sza
    https://doi.org/10.1075/tcb.00057.sza [Google Scholar]
  69. “UCLA Institute of Digital Research” 2021 FAQ How Do I Interpret a Regression Model When Some Variables Are Log Transformed? 2021 https://docs.google.com/document/d/1zHvX_0tOsbfAgz1VgizBh4WWJjQQxIvObYf0AP7Pd1U/edit
  70. Winter, Bodo
    2020Statistics for Linguists: An Introduction Using R. New York / London: Routledge.
    [Google Scholar]
  71. Ydewalle, Géry d’, Johan van Rensbergen, and Joris Pollet
    1987 “Reading a Message When the Same Message Is Available Auditorily in Another Language: The Case of Subtitling.” InEye Movements: From Physiology to Cognition, edited byJ. K. O’Regan and A. Levy-Schoen, 313–21. Amsterdam/New York: Elsevier. 10.1016/B978‑0‑444‑70113‑8.50047‑3
    https://doi.org/10.1016/B978-0-444-70113-8.50047-3 [Google Scholar]
  72. Ydewalle, Géry d’, Caroline Praet, Karl Verfaillie, and Johan Van Rensbergen
    1991 “Watching Subtitled Television: Automatic Reading Behavior.” Communication Research18 (5): 650–66. 10.1177/009365091018005005
    https://doi.org/10.1177/009365091018005005 [Google Scholar]
  73. Ydewalle, Géry d’, and Wim De Bruycker
    2007 “Eye Movements of Children and Adults While Reading Television Subtitles.” European Psychologist12 (3): 196–205. 10.1027/1016‑9040.12.3.196
    https://doi.org/10.1027/1016-9040.12.3.196 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/ts.21013.sil
Loading
/content/journals/10.1075/ts.21013.sil
Loading

Data & Media loading...

Keyword(s): audiovisual translation; confounding variables; eye tracking; linear mixed-effects models; LMMs; reading; subtitling; traditional statistical methods

Most Cited