1887
  1. Home
  2. e-Journals
  3. Linguistic Approaches to Bilingualism
  4. Online First Articles
  5. Online First Article
image of The art of wrangling
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Web-based eye-tracking is more accessible than ever. Researchers can now carry out visual world paradigm studies remotely and access never before tested, multilingual populations via the internet all without the need for an expensive eye-tracker. Web-based eye-tracking, however, requires careful experimental design and extensive data wrangling skills. In this paper, we provide a framework for reproducible, open science visual world paradigm studies using online experiments. We provide step-by-step instructions to building a typical visual world paradigm psycholinguistics study, and walk the reader through a series of data wrangling steps needed to prepare the data for visualization and analysis using the open-source software environment, R. Importantly, we highlight the key decisions researchers need to make and report in order to reproduce an analysis. We demonstrate our approach by carrying out a single change replication of an in-person eye-tracking study by Porretta et al. (2020). We conclude with best practices and recommendations for researchers carrying out bi-/multilingualism web-based visual world paradigm studies.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/lab.23071.bra
2024-08-13
2025-04-27

  • From This Site

    /content/journals/10.1075/lab.23071.bra
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K.
    (1998) Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, (), –. 10.1006/jmla.1997.2558
     https://doi.org/10.1006/jmla.1997.2558 [Google Scholar]
  2. Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K.
    (2019) Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods, (), –. 10.3758/s13428‑019‑01237‑x
     https://doi.org/10.3758/s13428-019-01237-x [Google Scholar]
  3. Apfelbaum, K. S., Klein-Packard, J., & McMurray, B.
    (2021) The pictures who shall not be named: Empirical support for benefits of preview in the visual world paradigm. Journal of Memory and Language, , 104279. 10.1016/j.jml.2021.104279
     https://doi.org/10.1016/j.jml.2021.104279 [Google Scholar]
  4. Barr, D. J.
    (2008) Analyzing ‘visual world’ eyetracking data using multilevel logistic regression. Journal of Memory and Language, (), –. 10.1016/j.jml.2007.09.002
     https://doi.org/10.1016/j.jml.2007.09.002 [Google Scholar]
  5. Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J.
    (2013) Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, (), –. 10.1016/j.jml.2012.11.001
     https://doi.org/10.1016/j.jml.2012.11.001 [Google Scholar]
  6. Bates, D., Mächler, M., Bolker, B., & Walker, S.
    (2014) Fitting linear Mixed-Effects models using lme4. 10.18637/jss.v067.i01
     https://doi.org/10.18637/jss.v067.i01 [Google Scholar]
  7. Bolibaugh, C., Vanek, N., & Marsden, E. J.
    (2021) Towards a credibility revolution in bilingualism research: Open data and materials as stepping stones to more reproducible and replicable research. Bilingualism: Language and Cognition, (), –. 10.1017/S1366728921000535
     https://doi.org/10.1017/S1366728921000535 [Google Scholar]
  8. Brysbaert, M., & Stevens, M.
    (2018) Power analysis and effect size in mixed effects models: A tutorial. Journal of Cognition, ():. 10.5334/joc.10
     https://doi.org/10.5334/joc.10 [Google Scholar]
  9. Brown, B., Tusmagambet, B., Rahming, V., Tu, C.-Y., DeSalvo, M. B., & Wiener, S.
    (2023) Searching for the “native” speaker: A preregistered conceptual replication and extension of Reid, Trofimovich, and O’Brien (2019). Applied Psycholinguistics, (), –. 10.1017/S0142716423000127
     https://doi.org/10.1017/S0142716423000127 [Google Scholar]
  10. Chen, M. C., Anderson, J. R., & Sohn, M. H.
    (2001) What can a mouse cursor tell us more?CHI ’01 Extended Abstracts on Human Factors in Computing Systems. 10.1145/634067.634234
     https://doi.org/10.1145/634067.634234 [Google Scholar]
  11. Cooper, R. M.
    (1974) The control of eye fixation by the meaning of spoken language. Cognitive Psychology, (), –. 10.1016/0010‑0285(74)90005‑X
     https://doi.org/10.1016/0010-0285(74)90005-X [Google Scholar]
  12. Coretta, S., Casillas, J. V., Roessig, S., Franke, M., Ahn, B., Al-Hoorie, A. H., Al-Tamimi, J., Alotaibi, N. E., AlShakhori, M. K., Altmiller, R. M., &
    (2023) Multidimensional signals and analytic flexibility: Estimating degrees of freedom in human-speech analyses. Advances in Methods and Practices in Psychological Science, (). 10.1177/25152459231162567
     https://doi.org/10.1177/25152459231162567 [Google Scholar]
  13. Cunnings, I., & Fujita, H.
    (2021) Quantifying individual differences in native and nonnative sentence processing. Applied Psycholinguistics, (), –. 10.1017/S0142716420000648
     https://doi.org/10.1017/S0142716420000648 [Google Scholar]
  14. Foster, E. D., & Deardorff, A.
    (2017) Open science framework (osf). Journal of the Medical Library Association, (). 10.5195/jmla.2017.88
     https://doi.org/10.5195/jmla.2017.88 [Google Scholar]
  15. Han, J., Kim, J., & Tsukada, K.
    (2023) Foreign accent in L1 (first language). Linguistic Approaches to Bilingualism. 10.1075/lab.22028.han
     https://doi.org/10.1075/lab.22028.han [Google Scholar]
  16. Huettig, F., & McQueen, J. M.
    (2007) The tug of war between phonological, semantic and shape information in language-mediated visual search. Journal of Memory and Language, (), –. 10.1016/j.jml.2007.02.001
     https://doi.org/10.1016/j.jml.2007.02.001 [Google Scholar]
  17. Ito, A., & Knoeferle, P.
    (2022) Analysing data from the psycholinguistic visual-world paradigm: Comparison of different analysis methods. Behavior Research Methods. https://link.springer.com/article/10.3758/s13428-022-01969-3. 10.3758/s13428‑022‑01969‑3
     https://doi.org/10.3758/s13428-022-01969-3 [Google Scholar]
  18. Kidd, E., Donnelly, S., & Christiansen, M. H.
    (2018) Individual differences in language acquisition and processing. Trends in Cognitive Sciences, (), –. 10.1016/j.tics.2017.11.006
     https://doi.org/10.1016/j.tics.2017.11.006 [Google Scholar]
  19. Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L.
    (2013) Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, (), –. 10.1016/j.jesp.2013.03.013
     https://doi.org/10.1016/j.jesp.2013.03.013 [Google Scholar]
  20. Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D.
    (2018) Replication in second language research: Narrative and systematic reviews and recommendations for the field. Language Learning, (), –. 10.1111/lang.12286
     https://doi.org/10.1111/lang.12286 [Google Scholar]
  21. Matin, E., Shao, K. C., & Boff, K. R.
    (1993) Saccadic overhead: Information-processing time with and without saccades. Perception & Psychophysics, (), –. 10.3758/BF03206780
     https://doi.org/10.3758/BF03206780 [Google Scholar]
  22. McMurray, B.
    (2023) I’m not sure that curve means what you think it means: Toward a [more] realistic understanding of the role of eye-movement generation in the visual world paradigm. Psychonomic Bulletin & Review, (), –. 10.3758/s13423‑022‑02143‑8
     https://doi.org/10.3758/s13423-022-02143-8 [Google Scholar]
  23. Milne, A. E., Bianco, R., Poole, K. C., Zhao, S., Oxenham, A. J., Billig, A. J., & Chait, M.
    (2021) An online headphone screening test based on dichotic pitch. Behavior Research Methods, (), –. 10.3758/s13428‑020‑01514‑0
     https://doi.org/10.3758/s13428-020-01514-0 [Google Scholar]
  24. Mirman, D., Dixon, J. A., & Magnuson, J. S.
    (2008) Statistical and computational models of the visual world paradigm: Growth curves and individual differences. Journal of Memory and Language, (), –. 10.1016/j.jml.2007.11.006
     https://doi.org/10.1016/j.jml.2007.11.006 [Google Scholar]
  25. Palan, S., & Schitter, C.
    (2018) Prolific.ac – a subject pool for online experiments. Journal of Behavioral and Experimental Finance, , –. 10.1016/j.jbef.2017.12.004
     https://doi.org/10.1016/j.jbef.2017.12.004 [Google Scholar]
  26. Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J., & Hays, J.
    (2016) Webgazer: Scalable webcam eye tracking using user interactions. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, –.
     [Google Scholar]
  27. Perpiñán, S. and Montrul, S.
    (2023) Does your regional variety help you acquire an additional language?Linguistic Approaches to Bilingualism, (), –. 10.1075/lab.22057.per
     https://doi.org/10.1075/lab.22057.per [Google Scholar]
  28. Porretta, V., Buchanan, L., & Järvikivi, J.
    (2020) When processing costs impact predictive processing: The case of foreign-accented speech and accent experience. Attention, Perception, & Psychophysics, (), –. 10.3758/s13414‑019‑01946‑7
     https://doi.org/10.3758/s13414-019-01946-7 [Google Scholar]
  29. Prystauka, Y., Altmann, G. T., & Rothman, J.
    (2023) Online eye tracking and real-time sentence processing: On opportunities and efficacy for capturing psycholinguistic effects of different magnitudes and diversity. Behavior Research Methods. 10.3758/s13428‑023‑02176‑4
     https://doi.org/10.3758/s13428-023-02176-4 [Google Scholar]
  30. R Core Team
    R Core Team (2022) R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria.
     [Google Scholar]
  31. Rodd, J. M.
    (in press). Moving experimental psychology online: How to obtain high quality data when we can’t see our participants. Journal of Memory and Language, (), . 10.1016/j.jml.2023.104472
     https://doi.org/10.1016/j.jml.2023.104472 [Google Scholar]
  32. Rothman, J., Bayram, F., DeLuca, V., Di Pisa, G., Duñabeitia, J. A., Gharibi, K., … Wulff, S.
    (2023) Monolingual comparative normativity in bilingualism research is out of “control”: Arguments and alternatives. Applied Psycholinguistics, (), –. 10.1017/S0142716422000315
     https://doi.org/10.1017/S0142716422000315 [Google Scholar]
  33. Semmelmann, K., & Weigelt, S.
    (2017) Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods, (), –. 10.3758/s13428‑017‑0913‑7
     https://doi.org/10.3758/s13428-017-0913-7 [Google Scholar]
  34. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C.
    (1995) Integration of visual and linguistic information in spoken language comprehension. Science, (), –. 10.1126/science.7777863
     https://doi.org/10.1126/science.7777863 [Google Scholar]
  35. Vos, M., Minor, S., & Ramchand, G. C.
    (2022) Comparing infrared and webcam eye tracking in the visual world paradigm. Glossa Psycholinguistics, (). 10.5070/G6011131
     https://doi.org/10.5070/G6011131 [Google Scholar]
  36. Wickham, H.
    (2014) Tidy data. Journal of Statistical Software, (). 10.18637/jss.v059.i10
     https://doi.org/10.18637/jss.v059.i10 [Google Scholar]
  37. Wickham, H., & Grolemund, G.
    (2017, January). R for data science: Import, tidy, transform, visualize, and model data (1st ed.). O’Reilly Media. r4ds.had.co.nz/
     [Google Scholar]
  38. Wood, S.
    (2017) Generalized additive models: An introduction with R (2nd ed.). Chapman; Hall/CRC. 10.1201/9781315370279
     https://doi.org/10.1201/9781315370279 [Google Scholar]
/content/journals/10.1075/lab.23071.bra
Loading
The art of wrangling
John Benjamins ; https://doi.org/10.1075/lab.23071.bra
/content/journals/10.1075/lab.23071.bra
/content/journals/10.1075/lab.23071.bra
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: visual world paradigm ; web-based eye-tracking ; open science ; replication ; data quality

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error