1887
image of A note on the modeling of the effects of experimental time in psycholinguistic experiments

Abstract

Abstract

called attention to problems that arise when chronometric experiments implementing specific factorial designs are analysed with the generalized additive mixed model (GAMM), using factor smooths to capture trial-to-trial dependencies. From a series of simulations incorporating such dependencies, they conclude that GAMMs are inappropriate for between-subject designs. They argue that in addition GAMMs come with too many modeling possibilities, and advise using the linear mixed model (LMM) instead. As clarified by the title of their paper, their conclusion is: “Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit”.

We address the questions raised by , who clearly demonstrated that problems can indeed arise when using factor smooths in combination with factorial designs. We show that the problem does not arise when using by-smooths. Furthermore, we have traced a bug in the implementation of factor smooths in the package, which will have been removed from version 1.8–36 onwards.

To illustrate that GAMMs now produce correct estimates, we report simulation studies implementing different by-subject longitudinal effects. The maximal LMM emerges as slightly conservative compared to GAMMs, and GAMMs provide estimated coefficients that can be less variable across simulation runs. We also discuss two datasets where time-varying effects interact with numerical predictors in a theoretically informative way.

Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/ml.21012.baa
2022-04-04
2022-05-27
Loading full text...

Full text loading...

/deliver/fulltext/10.1075/ml.21012.baa/ml.21012.baa.html?itemId=/content/journals/10.1075/ml.21012.baa&mimeType=html&fmt=ahah

References

  1. Baayen, R. H.
    (2010) The directed compound graph of English. an exploration of lexical connectivity and its processing consequences. InOlsen, S., editor, New impulses in word-formation (Linguistische Berichte Sonderheft 17), pages383–402. Buske, Hamburg.
    [Google Scholar]
  2. Baayen, R. H., van Rij, J., de Cat, C., and Wood, S. N.
    (2017a) Autocorrelated errors in experimental data in the language sciences: Some solutions offered by generalized additive mixed models. InSpeelman, D., Heylen, K., and Geeraerts, D., editors, Mixed Effects Regression Models in Linguistics, pages49–69. Springer, Berlin.
    [Google Scholar]
  3. Baayen, R. H., Vasishth, S., Bates, D., and Kliegl, R.
    (2017b) The cave of shadows. Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94:206–234. 10.1016/j.jml.2016.11.006
    https://doi.org/10.1016/j.jml.2016.11.006 [Google Scholar]
  4. Barr, D. J., Levy, R., Scheepers, C., and Tily, H. J.
    (2013) Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3):255–278. 10.1016/j.jml.2012.11.001
    https://doi.org/10.1016/j.jml.2012.11.001 [Google Scholar]
  5. Bates, D. M., Kliegl, R., Vasishth, S., and Baayen, R. H.
    (2015) Parsimonious mixed models. arXiv.org.
    [Google Scholar]
  6. Chuang, Y.-Y.
    (2017) The effect of phonetic variation on word recognition in Taiwan Mandarin. PhD thesis, National Taiwan University, Taipei.
    [Google Scholar]
  7. Chuang, Y.-Y., Fon, J., Papakyritsis, I., and Baayen, R. H.
    (2021) Analyzing phonetic data with generalized additive mixed models. InBall, M. J., editor, Manual of Clinical Phonetics. Routledge. 10.4324/9780429320903‑10
    https://doi.org/10.4324/9780429320903-10 [Google Scholar]
  8. Francis, G.
    (2012) Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review, 19(6):975–991. 10.3758/s13423‑012‑0322‑y
    https://doi.org/10.3758/s13423-012-0322-y [Google Scholar]
  9. Harm, M. W. and Seidenberg, M. S.
    (2004) Computing the meanings of words in reading: Cooperative division of labor between visual and phonological processes. Psychological Review, 111:662–720. 10.1037/0033‑295X.111.3.662
    https://doi.org/10.1037/0033-295X.111.3.662 [Google Scholar]
  10. Hastie, T. and Tibshirani, R.
    (1990) Generalized Additive Models. Chapman & Hall, London.
    [Google Scholar]
  11. Keuleers, E., Lacey, P., Rastle, K., and Brysbaert, M.
    (2012) The British Lexicon Project: Lexical decision data for 28,730 monosyllabic and disyllabic English words. Behavior Research Methods, 44:287–304. 10.3758/s13428‑011‑0118‑4
    https://doi.org/10.3758/s13428-011-0118-4 [Google Scholar]
  12. Matuschek, H., Kliegl, R., Vasishth, S., Baayen, R. H., and Bates, D. M.
    (2017) Balancing Type I Error and Power in Linear Mixed Models. Journal of Memory and Language. 10.1016/j.jml.2017.01.001
    https://doi.org/10.1016/j.jml.2017.01.001 [Google Scholar]
  13. Mock, P., Tibus, M., Ehlis, A.-C., Baayen, H., and Gerjets, P.
    (2018) Predicting adhd risk from touch interaction data. InProceedings of the 20th ACM International Conference on Multimodal Interaction, pages446–454. 10.1145/3242969.3242986
    https://doi.org/10.1145/3242969.3242986 [Google Scholar]
  14. Open Science Collaboration
    Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science, 349(6251):aac4716. 10.1126/science.aac4716
    https://doi.org/10.1126/science.aac4716 [Google Scholar]
  15. Pinheiro, J. C. and Bates, D. M.
    (2000) Mixed-effects models in S and S-PLUS. Statistics and Computing. Springer, New York. 10.1007/978‑1‑4419‑0318‑1
    https://doi.org/10.1007/978-1-4419-0318-1 [Google Scholar]
  16. Segalowitz, N. S. and Segalowitz, S. J.
    (1993) Skilled performance, practice, and the differentiation of speed-up from automatization effects: Evidence from second language word recognition. Applied Psycholinguistics, 14(03):369–385. 10.1017/S0142716400010845
    https://doi.org/10.1017/S0142716400010845 [Google Scholar]
  17. Thul, R., Conklin, K., and Barr, D. J.
    (2020) Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Open Science Framework. downloadedOctober 2020. 10.31234/osf.io/ywkeq
    https://doi.org/10.31234/osf.io/ywkeq [Google Scholar]
  18. (2021) Using gamms to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit. Journal of Memory and Language, 120:104247. 10.1016/j.jml.2021.104247
    https://doi.org/10.1016/j.jml.2021.104247 [Google Scholar]
  19. van Rij, J., Hendriks, P., van Rijn, H., Baayen, R. H., and Wood, S. N.
    (2019) Analyzing the time course of pupillometric data. Trends in hearing, 23:2331216519832483. 10.1177/2331216519832483
    https://doi.org/10.1177/2331216519832483 [Google Scholar]
  20. Westfall, J., Kenny, D. A., and Judd, C. M.
    (2014) Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5):2020. 10.1037/xge0000014
    https://doi.org/10.1037/xge0000014 [Google Scholar]
  21. Wieling, M.
    (2018) Analyzing dynamic phonetic data using generalized additive mixed modeling: a tutorial focusing on articulatory differences between l1 and l2 speakers of english. Journal of Phonetics, 70:86–116. 10.1016/j.wocn.2018.03.002
    https://doi.org/10.1016/j.wocn.2018.03.002 [Google Scholar]
  22. Wood, S. N.
    (2013) On p-values for smooth components of an extended generalized additive model. Biometrika, 100:221–228. 10.1093/biomet/ass048
    https://doi.org/10.1093/biomet/ass048 [Google Scholar]
  23. (2016) Just Another Gibbs Additive Modeller: Interfacing JAGS and mgcv. arXiv preprint arXiv:1602.02539v1.
    [Google Scholar]
  24. (2017) Generalized Additive Models. Chapman & Hall/CRC, New York. 10.1201/9781315370279
    https://doi.org/10.1201/9781315370279 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/ml.21012.baa
Loading
/content/journals/10.1075/ml.21012.baa
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error