Abstract
Abstract
In this paper, we aim to promote the use of linear mixed models (LMMs) in eye-tracking research on subtitling. Using eye tracking to study viewers’ reading of subtitles often warrants controlling for many confounding variables. However, even assuming that these variables are known to researchers, such control may not be possible or desired. Traditional statistical methods such as t-tests or ANOVAs exacerbate the problem due to the use of aggregated data: each participant has one data point per dependent variable. As a solution, we propose the use of LMMs, which are better suited to account for a number of subtitle and participant characteristics, thus explaining more variance. We introduce essential theoretical aspects of LMMs and highlight some of their advantages over traditional statistical methods. To illustrate our point, we compare two analyses of the same dataset: one using a t-test; another using LMMs.
© John Benjamins Publishing Company
Article metrics loading...
/content/journals/10.1075/ts.21013.sil
2022-06-14
2024-03-29
-
/content/journals/10.1075/ts.21013.sil
dcterms_title,dcterms_subject,pub_keyword
-contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
10
5
Full text loading...
References
-
Baayen, Harald
2008a Analyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge: Cambridge University Press.
10.1017/CBO9780511801686
https://doi.org/10.1017/CBO9780511801686
[Google Scholar]
-
Baayen, Harald
2008b Exploratory Data Analysis: An Introduction to R for the Language Sciences. Cambridge: Cambridge University Press.
[Google Scholar]
-
Balling, Laura Winther
2008 “A Brief Introduction to Regression Designs and Mixed-Effects Modelling by a Recent Convert.” Copenhagen Studies in Language, 175–92.
[Google Scholar]
-
Balling, Laura Winther, and Kristian Tangsgaard Hvelplund
2015 “Design and Statistics in Quantitative Translation (Process) Research”.
Translation Spaces4 (1): 170–187.
10.1075/ts.4.1.08bal
https://doi.org/10.1075/ts.4.1.08bal
[Google Scholar]
-
Barr, Dale. J., Roger Levy, Christoph Scheepers, and Harry J. Tily
2013 “Random Effects Structure for Confirmatory Hypothesis Testing: Keep It Maximal.”
J Mem Lang68 (3), 255–278.
10.1016/j.jml.2012.11.001
https://doi.org/10.1016/j.jml.2012.11.001
[Google Scholar]
-
Bates, Douglas, Reinhold Kliegl, Shravan Vasishth, and R. Harald Baayen
2018 “Parsimonius Mixed Models.” arXiv:1506.04967v2 [stat.ME].
[Google Scholar]
-
Bisson, Marie-Josée, Walter J. B. Heuven, Kathy Conklin, and Richard J. Tunney
2014 “The Role of Repeated Exposure to Multimodal Input in Incidental Acquisition of Foreign Language Vocabulary.”
Language Learning64 (4): 855–77.
10.1111/lang.12085
https://doi.org/10.1111/lang.12085
[Google Scholar]
-
Bolker, Ben
-
Brown, Violet A.
2021 An Introduction to Linear Mixed-Effects Modelling in R.
Advances in Methods and Practices in Psychological Science4 (1): 1–19.
10.1177/2515245920960351
https://doi.org/10.1177/2515245920960351
[Google Scholar]
-
Brysbaert, Marc, and Boris New
2009 “Moving beyond Kučera and Francis: A Critical Evaluation of Current Word Frequency Norms and the Introduction of a New and Improved Word Frequency Measure for American English.”
Behavior Research Methods41 (4): 977–90.
10.3758/BRM.41.4.977
https://doi.org/10.3758/BRM.41.4.977
[Google Scholar]
-
Burnham, Denis, Kaoru Sekiyama, Gerard Bailly, Pascal Perrier, and Eric Vatikiotis-Bateson
2012 “Investigating Auditory-Visual Speech Perception Development.” In
Audiovisual Speech Processing, edited by
Gérard Bailly, Pascal Perrier and Eric Vatikiotis-Bateson, 62–75. Cambridge: Cambridge University Press.
10.1017/CBO9780511843891.006
https://doi.org/10.1017/CBO9780511843891.006
[Google Scholar]
-
Cambra, Cristina, Olivier Penacchio, Núria Silvestre, and Aurora Leal
2014 “Visual Attention to Subtitles When Viewing a Cartoon by Deaf and Hearing Children: An Eye-Tracking Pilot Study.”
Perspectives22 (4): 607–17.
10.1080/0907676X.2014.923477
https://doi.org/10.1080/0907676X.2014.923477
[Google Scholar]
-
Carson, Robyn J., and Christina M. L. Beeson
2013 “Crossing Language Barriers: Using Crossed Random Effects Modelling in Psycholinguistics Research.”
Tutorials in Quantitative Methods for Psychology9 (1): 25–41.
10.20982/tqmp.09.1.p025
https://doi.org/10.20982/tqmp.09.1.p025
[Google Scholar]
-
Crossley, Scott A., Tom Cobb, and Danielle S. McNamara
2013 “Comparing Count-Based and Band-Based Indices of Word Frequency: Implications for Active Vocabulary Research and Pedagogical Applications.”
System41 (4): 965–81.
10.1016/j.system.2013.08.002
https://doi.org/10.1016/j.system.2013.08.002
[Google Scholar]
-
Cunnings, Ian, and Ian Finlayson
2015 “Mixed Effects Modeling and Longitudinal Data Analysis.” In
Advancing Quantitative Methods in Second Language Research, edited by
Luke Plonsky, 159–81. New York: Routledge.
10.4324/9781315870908‑8
https://doi.org/10.4324/9781315870908-8
[Google Scholar]
-
De Bruycker, Wim, and Gery d’Ydewalle
2003 “Reading Native and Foreign Language Television Subtitles in Children and Adults.” In
The Mind’s Eye: Cognitive and Applied Aspects of Eye Movement Research, edited by
Jukka Hyönä, Ralph Radach, and Heiner Deubel, 671–84. Amsterdam: North-Holland.
10.1016/B978‑044451020‑4/50036‑0
https://doi.org/10.1016/B978-044451020-4/50036-0
[Google Scholar]
-
Dijkstra, Ton, Koji Miwa, Bianca Brummelhuis, Maya Sappelli, and Harald Baayen
2010 “How Cross-Language Similarity and Task Demands Affect Cognate Recognition.”
Journal of Memory and Language62 (3): 284–301.
10.1016/j.jml.2009.12.003
https://doi.org/10.1016/j.jml.2009.12.003
[Google Scholar]
-
Doherty, Stephen, and Jan-Louis Kruger
2018 “The Development of Eye Tracking in Empirical Research on Subtitling and Captioning.” In
Seeing into Screens: Eye Tracking and the Moving Imaging, edited by
Jodi Sita, Tessa Dwyer, Sean Redmond, and Claire Perkins, 46–64. London: Bloomsbury.
10.5040/9781501329012.0009
https://doi.org/10.5040/9781501329012.0009
[Google Scholar]
-
Duyck, Wouter, Eva Van Assche, Denis Drieghe, and Robert J. Hartsuiker
2007 “Visual Word Recognition by Bilinguals in a Sentence Context: Evidence for Nonselective Lexical Access.”
J Exp Psychol Learn Mem Cogn33 (4): 663–79.
10.1037/0278‑7393.33.4.663
https://doi.org/10.1037/0278-7393.33.4.663
[Google Scholar]
-
Díaz-Cintas, Jorge and Aline Remael
(2021) Subtitling. Concepts and Practices. London and New York, Routledge.
[Google Scholar]
-
Ellis, Nick C., and Alan Beaton
https://doi.org/10.1111/j.1467-1770.1993.tb00627.x
[Google Scholar]
-
Field, Andy
2017 Discovering Statistics Using IBM SPSS Statistics. 5th ed.London: SAGE Publications, Inc.
[Google Scholar]
-
Garson, G. David
2020 Multilevel modelling: Applications in Stata®, IBM® SPSS®, SAS®, R, & HLMTM. New York: Sage.
[Google Scholar]
-
Gerber-Morón, Olivia, and Agnieszka Szarkowska
2018 “Line Breaks in Subtitling: An Eye Tracking Study on Viewer Preferences.”
Journal of Eye Movement Research11 (3): 1–22.
10.16910/jemr.11.3.2
https://doi.org/10.16910/jemr.11.3.2
[Google Scholar]
-
Gerber-Morón, Olivia, Agnieszka Szarkowska, and Bencie Woll
2018 “The Impact of Text Segmentation on Subtitle Reading.”
Journal of Eye Movement Research11 (4): 1–18.
10.16910/11.4.2
https://doi.org/10.16910/11.4.2
[Google Scholar]
-
Ghia, Elisa
2012 “The Impact of Translation Strategies on Subtitle Reading.” InEye Tracking in Audiovisual Translation, edited byElisa Perego, 155–82. Roma: Aracne Editrice.
[Google Scholar]
-
Gile, Daniel
2016 “Experimental research.” InResearching Translation and Interpreting, edited byClaudia V. Angelelli and Brian James Baer, 220–228. New York: Routledge.
[Google Scholar]
-
Hajduk, Gabriela K.
-
Harrison, Xavier. A., Lynda Donaldson, Maria E. Correa-Cano, Julian Evans, David N. Fisher, Cecily E. D. Goodwin, Beth. S. Robinson, David J. Hodgson, and Richard Inger
2018 “A Brief Introduction to Mixed Effects Modelling and Multi-Model Inference in Ecology.”
PeerJ61: e4794.
10.7717/peerj.4794
https://doi.org/10.7717/peerj.4794
[Google Scholar]
-
Heck, Ronald H., Scott L. Thomas, and Lynn N. Tabata
2012 Multilevel Modelling of Categorical Outcomes Using IBM SPSS. Quantitative Methodology Series. New York / London: Routledge.
[Google Scholar]
-
Hedeker, Donald
2003 “A Mixed-Effects Multinomial Logistic Regression Model.”
Statistics in Medicine22 (9): 1433–46.
10.1002/sim.1522
https://doi.org/10.1002/sim.1522
[Google Scholar]
-
Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost van de Weijer
2011 Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.
[Google Scholar]
-
Inhoff, Albrecht W., and Keith Rayner
1986 “Parafoveal Word Processing during Eye Fixations in Reading: Effects of Word Frequency.”
Perception & Psychophysics40 (6): 431–39.
10.3758/BF03208203
https://doi.org/10.3758/BF03208203
[Google Scholar]
-
Jensema, Carl
1998 “Viewer Reaction to Different Television Captioning Speeds.”
American Annals of the Deaf143 (4): 318–24.
10.1353/aad.2012.0073
https://doi.org/10.1353/aad.2012.0073
[Google Scholar]
-
Koolstra, Cees M., Tom H. A. Van Der Voort, and Gery d’Ydewalle
1999 “Lengthening the Presentation Time of Subtitles on Television: Effects on Children’s Reading Time and Recognition.”
Communications24 (4): 407–22.
10.1515/comm.1999.24.4.407
https://doi.org/10.1515/comm.1999.24.4.407
[Google Scholar]
-
Krejtz, Izabela, Agnieszka Szarkowska, and Maria Łogińska
2016 “Reading Function and Content Words in Subtitled Videos.”
Journal Of Deaf Studies And Deaf Education21 (2): 222–32.
10.1093/deafed/env061
https://doi.org/10.1093/deafed/env061
[Google Scholar]
-
Kruger, Jan-Louis
2018 “Eye tracking in audiovisual translation research”. In
The Routledge Handbook of Audiovisual Translation, edited by
L. Pérez-González, 350–366. Routledge.
10.4324/9781315717166‑22
https://doi.org/10.4324/9781315717166-22
[Google Scholar]
-
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew
2014 “Attention Distribution and Cognitive Load in a Subtitled Academic Lecture: L1 vs. L2.”
Journal of Eye Movement Research7 (5): 1–15.
10.16910/jemr.7.5.4
https://doi.org/10.16910/jemr.7.5.4
[Google Scholar]
-
Kruger, Jan-Louis, Stephen Doherty, and María T. Soto-Sanfiel
2017 “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.”
Comunicar, 501 (January): 23–32.
10.3916/C50‑2017‑02
https://doi.org/10.3916/C50-2017-02
[Google Scholar]
-
Kruger, Jan-Louis, and Stephen Doherty
2018 “Triangulation of Online and Offline Measures of Processing and Reception in AVT.” In
Reception Studies and Audiovisual Translation, edited by
Elena Di Giovanni and Yves Gambier, 91–109. Amsterdam/New York: John Benjamins.
10.1075/btl.141.06kru
https://doi.org/10.1075/btl.141.06kru
[Google Scholar]
-
Lemhöfer, Kristin, and Ton Dijkstra
2004 “Recognizing Cognates and Interlingual Homographs: Effects of Code Similarity in Language-Specific and Generalized Lexical Decision.”
Memory & Cognition32 (4): 533–50.
10.3758/BF03195845
https://doi.org/10.3758/BF03195845
[Google Scholar]
-
Liao, Sixin, Jan-Louis Kruger, and Stephen Doherty
2020 “The Impact of Monolingual and Bilingual Subtitles on Visual Attention, Cognitive Load, and Comprehension.” Journal of Specialised Translation331: 70–98.
[Google Scholar]
-
Liao, Sixin, Lili Yu, Erik D. Reichle, and Jan-Louis Kruger
2020 “Using Eye Movements to Study the Reading of Subtitles in Video.”
Scientific Studies of Reading25 (5): 417–435.
10.1080/10888438.2020.1823986
https://doi.org/10.1080/10888438.2020.1823986
[Google Scholar]
-
Łuczak, Krzysztof
2017 “The Effects of the Language of the Soundtrack on Film Comprehension, Cognitive Load and Subtitle Reading Patterns. An Eye-Tracking Study.” Institite of Applied Linguistics. Warsaw: University of Warsaw.
[Google Scholar]
-
Mellinger, Christopher D., and Thomas A. Hanson
(2017) Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.
[Google Scholar]
-
Meteyard, Lotte, and Robert A. I. Davies
2020 “Best Practice Guidance for Linear Mixed-Effects Models in Psychological Science.”
Journal of Memory and Language1121.
10.1016/j.jml.2020.104092
https://doi.org/10.1016/j.jml.2020.104092
[Google Scholar]
-
Moran, Siobhan
2009 “The Effect of Linguistic Variation on Subtitle Reception.” Toronto: York University.
[Google Scholar]
-
Muñoz, Carmen
https://doi.org/10.1016/j.system.2017.04.015
[Google Scholar]
-
Orrego-Carmona, David
2015 “The Reception of (Non)Professional Subtitling.” Department of English and German Studies. Tarragona: Universitat Rovira i Virgili.
[Google Scholar]
-
Peters, Elke
2019 “Factors Affecting the Learning of Single-Word Items.” In
The Routledge Handbook of Vocabulary Studies, edited by
Stuart Webb, 125–42. New York: Routledge.
10.4324/9780429291586‑9
https://doi.org/10.4324/9780429291586-9
[Google Scholar]
-
Ragni, Valentina
2020 “More than Meets the Eye: An Eye-Tracking Study of the Effects of Translation on the Processing and Memorisation of Reversed Subtitles.” Journal of Specialised Translation331: 99–128.
[Google Scholar]
-
Rajendran, Dhevi J., Andrew T. Duchowski, Pilar Orero, Juan Martínez, and Pablo Romero-Fresco
2013 “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.”
Perspectives21 (1): 5–21.
10.1080/0907676X.2012.722651
https://doi.org/10.1080/0907676X.2012.722651
[Google Scholar]
-
Rayner, Keith
2009 “Eye movements and attention in reading, scene perception, and visual search”.
The Quarterly Journal of Experimental Psychology62(8); 1457–1506.
10.1080/17470210902816461
https://doi.org/10.1080/17470210902816461
[Google Scholar]
-
Rayner, Keith, and Susan A. Duffy
1986 “Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity.”
Memory & Cognition14 (3): 191–201.
10.3758/BF03197692
https://doi.org/10.3758/BF03197692
[Google Scholar]
-
Rayner, Keith, Tessa Warren, Barbara J. Juhasz, and Simon P. Liversedge
2004 “The Effect of Plausibility on Eye Movements in Reading.”
J Exp Psychol Learn Mem Cogn30 (6): 1290–1301.
10.1037/0278‑7393.30.6.1290
https://doi.org/10.1037/0278-7393.30.6.1290
[Google Scholar]
-
Rayner, K., Timothy J. Slattery, Denis Drieghe, and Simon P. Liversedge
2011 “Eye Movements and Word Skipping during Reading: Effects of Word Length and Predictability.”
J Exp Psychol Hum Percept Perform37 (2): 514–28.
10.1037/a0020990
https://doi.org/10.1037/a0020990
[Google Scholar]
-
Romero-Fresco, Pablo
2015 “Final Thoughts: Viewing Speed in Subtitling.” InThe Reception of Subtitles for the Deaf and Hard of Hearing in Europe, edited byPablo Romero-Fresco, 335–41. Bern: Peter Lang.
[Google Scholar]
-
RStudio Team
RStudio Team
2021 Studio: Integrated Development for R. RStudio. R. Boston.
www.rstudio.com/
[Google Scholar]
-
Saldanha, Gabriela, and Sharon O’Brien
https://doi.org/10.4324/9781315760100
[Google Scholar]
-
Schwartz, Ana I., and Judith F. Kroll
2006 “Bilingual Lexical Activation in Sentence Context.”
Journal of Memory and Language55 (2): 197–212.
10.1016/j.jml.2006.03.004
https://doi.org/10.1016/j.jml.2006.03.004
[Google Scholar]
-
Singmann, Henrik, and David Kellen
2019 “An Introduction to Mixed Models for Experimental Psychology.” In
New Methods in Cognitive Psychology, edited by
Daniel Spieler and Eric Schumacher, 1st ed., 4–31. Routledge.
10.4324/9780429318405‑2
https://doi.org/10.4324/9780429318405-2
[Google Scholar]
-
Silva, Breno B., David Orrego-Carmona, and Agnieszka Szarkowska
2021a Data for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’
Figshare. Available at: (accessed
March 2022).
10.6084/m9.figshare.14403389.v1
https://doi.org/10.6084/m9.figshare.14403389.v1
[Google Scholar]
-
Silva, Breno B., David Orrego-Carmona, and Agnieszka Szarkowska
2021b Appendices for paper entitled ‘Using linear mixed models to analyze data from eye-tracking research on subtitling’
Figshare. Available at: (accessed
March 2022).
10.1075/ts.21013.sil
https://doi.org/10.1075/ts.21013.sil
[Google Scholar]
-
Szarkowska, Agnieszka, Izabela Krejtz, Zuzanna Kłyszejko, and Anna Wieczorek
2011 “Verbatim, Standard, or Edited? Reading Patterns of Different Captioning Styles among Deaf, Hard of Hearing, and Hearing Viewers.”
American Annals of the Deaf156 (4): 363–78.
10.1353/aad.2011.0039
https://doi.org/10.1353/aad.2011.0039
[Google Scholar]
-
Szarkowska, Agnieszka, and Olivia Gerber-Morón
https://doi.org/10.1371/journal.pone.0199331
[Google Scholar]
-
Szarkowska, Agnieszka, and Olivia Gerber-Morón
2019 “Two or Three Lines: A Mixed-Methods Study on Subtitle Processing and Preferences.”
Perspectives27 (1): 144–64.
10.1080/0907676X.2018.1520267
https://doi.org/10.1080/0907676X.2018.1520267
[Google Scholar]
-
Szarkowska, Agnieszka, and Lidia Bogucka
2019 “Six-Second Rule Revisited: An Eye-Tracking Study on the Impact of Speech Rate and Language Proficiency on Subtitle Reading.”
Translation, Cognition & Behavior2 (1): 101–24.
10.1075/tcb.00022.sza
https://doi.org/10.1075/tcb.00022.sza
[Google Scholar]
-
Szarkowska, Agnieszka, Breno B. Silva, and David Orrego-Carmona
2021 Effects of subtitle speed on proportional reading time: Re-analysing subtitle reading data with mixed effects models.
Translation, Cognition & Behavior, 4(2): 305–330.
10.1075/tcb.00057.sza
https://doi.org/10.1075/tcb.00057.sza
[Google Scholar]
-
UCLA Institute of Digital Research
-
Winter, Bodo
2020 Statistics for Linguists: An Introduction Using R. New York / London: Routledge.
[Google Scholar]
-
Ydewalle, Géry d’, Johan van Rensbergen, and Joris Pollet
1987 “Reading a Message When the Same Message Is Available Auditorily in Another Language: The Case of Subtitling.” In
Eye Movements: From Physiology to Cognition, edited by
J. K. O’Regan and A. Levy-Schoen, 313–21. Amsterdam/New York: Elsevier.
10.1016/B978‑0‑444‑70113‑8.50047‑3
https://doi.org/10.1016/B978-0-444-70113-8.50047-3
[Google Scholar]
-
Ydewalle, Géry d’, Caroline Praet, Karl Verfaillie, and Johan Van Rensbergen
1991 “Watching Subtitled Television: Automatic Reading Behavior.”
Communication Research18 (5): 650–66.
10.1177/009365091018005005
https://doi.org/10.1177/009365091018005005
[Google Scholar]
-
Ydewalle, Géry d’, and Wim De Bruycker
2007 “Eye Movements of Children and Adults While Reading Television Subtitles.”
European Psychologist12 (3): 196–205.
10.1027/1016‑9040.12.3.196
https://doi.org/10.1027/1016-9040.12.3.196
[Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/ts.21013.sil