1887
  1. Home
  2. e-Journals
  3. International Journal of Corpus Linguistics
  4. Online First Articles
  5. Online First Article
image of Not all linguistic variation is equally predictable
EPUB
PDF

Abstract

We compare to what extent the choice of a variant can be predicted from language-internal factors for two linguistic variables: the English dative alternation and the omission of the infinitival marker in a Swedish future-tense construction. Previous research has shown that for the dative alternation, near-ceiling performance can be achieved by fitting a regression model with manually selected predictors. Similar attempts for -omission have been unsuccessful. To test whether the two variables differ in predictability or whether optimal methods have not been found for -omission, we apply a large language model (LLM) to the same task. For the dative alternation, LLM and regression perform equally well. For -omission, LLM outperforms regression, but still performs worse than for the dative alternation, thus suggesting that -omission is inherently less predictable. We argue that LLMs can be useful for estimating how much the choice of a variant depends on language-internal factors.

© John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/ijcl.24065.mor
2026-06-16
2026-07-17

  • From This Site

    /content/journals/10.1075/ijcl.24065.mor
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/10.1075/ijcl.24065.mor/ijcl.24065.mor.html?itemId=/content/journals/10.1075/ijcl.24065.mor&mimeType=html&fmt=ahah

References

  1. Agrawal, M., Peterson, J. C., & Griffiths, T. L.
    (2020) Scaling up psychology via scientific regret minimization. Proceedings of the National Academy of Sciences, (), –. 10.1073/pnas.1915841117
     https://doi.org/10.1073/pnas.1915841117 [Google Scholar]
  2. Arnold, J. E., Losongco, A., Wasow, T., & Ginstrom, R.
    (2000) Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering. Language, (), –. 10.2307/417392
     https://doi.org/10.2307/417392 [Google Scholar]
  3. Baayen, R. H.
    (2011) Corpus linguistics and naive discriminative learning. Revista Brasileira de Linguística Aplicada, (), –. 10.1590/S1984‑63982011000200003
     https://doi.org/10.1590/S1984-63982011000200003 [Google Scholar]
  4. (2024) The wompom. Corpus Linguistics and Linguistic Theory, 20(), –. 10.1515/cllt‑2024‑0053
     https://doi.org/10.1515/cllt-2024-0053 [Google Scholar]
  5. Bates, D., Mächler, M., Bolker, B., & Walker, S.
    (2015) Fitting linear mixed-effects models using lme4. Journal of Statistical Software, (), –. 10.18637/jss.v067.i01
     https://doi.org/10.18637/jss.v067.i01 [Google Scholar]
  6. Berdicevskis, A., Bouma, G., Kurtz, R., Morger, F., Öhman, J., Adesam, Y., Borin, L., Dannélls, D., Forsberg, M., Isbister, T., Lindahl, A., Malmsten, M., Rekathati, F., Sahlgren, M., Volodina, E., Börjeson, L., Hengchen, S., & Tahmasebi, N.
    (2023) Superlim: A Swedish language understanding evaluation benchmark. InH. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 conference on empirical methods in Natural Language Processing (pp.–). Association for Computational Linguistics. 10.18653/v1/2023.emnlp‑main.506
     https://doi.org/10.18653/v1/2023.emnlp-main.506 [Google Scholar]
  7. Berdicevskis, A., Coussé, E., Koplenig, A., & Adesam, Y.
    (2024) To drop or not to drop? Predicting the omission of the infinitival marker in a Swedish future construction. Corpus Linguistics and Linguistic Theory, (), –. 10.1515/cllt‑2022‑0101
     https://doi.org/10.1515/cllt-2022-0101 [Google Scholar]
  8. Bresnan, J., Anna, C., Nikitina, T., & Baayen, R. H.
    (2007) Predicting the dative alternation. InG. Bouma, I. Krämer, & J. Zwarts (Eds.), Cognitive foundations of interpretation (pp.–). KNAW.
     [Google Scholar]
  9. Bresnan, J., & Ford, M.
    (2010) Predicting syntax: Processing dative constructions in American and Australian varieties of English. Language, (), –. https://www.jstor.org/stable/40666302
     [Google Scholar]
  10. Bresnan, J., Rosenbach, A., Szmrecsanyi, B., Tagliamonte, S., & Simon, T.
    (2017) Syntactic alternations data: Datives and genitives in four varieties of English. Stanford Digital Repository. purl.stanford.edu/qj187zs3852.
     [Google Scholar]
  11. Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M.
    (2010) The balanced accuracy and its posterior distribution. InJ. E. Guerrero (ed.) 2010 20th international conference on pattern recognition (pp.–). Institute of Electrical and Electronics Engineers. 10.1109/ICPR.2010.764
     https://doi.org/10.1109/ICPR.2010.764 [Google Scholar]
  12. Chambaz, A., & Desagulier, G.
    (2016) Predicting is not explaining: Targeted learning of the dative alternation. Journal of Causal Inference, (), –. 10.1515/jci‑2014‑0037
     https://doi.org/10.1515/jci-2014-0037 [Google Scholar]
  13. Daelemans, W., Zavrel, J., Van der Sloot, K., & Van den Bosch, A.
    (2010) TiMBL: Tilburg memory based learner reference guide. Version 6.3 (Technical Report No. ILK 10–01). Computational Linguistics Tilburg University. https://languagemachines.github.io/timbl/.
     [Google Scholar]
  14. Devlin, J., Chang, M. -W., Lee, K., & Toutanova, K.
    (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. InJ. Burstein, C. Doran, & T. Solorio (Eds.), Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, Volume 1 (Long and Short Papers) (pp.–). 10.48550/arXiv.1810.04805
     https://doi.org/10.48550/arXiv.1810.04805 [Google Scholar]
  15. Dyer, W., Torres, C., Scontras, G., & Futrell, R.
    (2023) Evaluating a century of progress on the cognitive science of adjective ordering. Transactions of the Association for Computational Linguistics, , –. 10.1162/tacl_a_00596
     https://doi.org/10.1162/tacl_a_00596 [Google Scholar]
  16. Eckert, P.
    (2012) Three waves of variation study: The emergence of meaning in the study of sociolinguistic variation. Annual Review of Anthropology, , –. 10.1146/annurev‑anthro‑092611‑145828
     https://doi.org/10.1146/annurev-anthro-092611-145828 [Google Scholar]
  17. Engel, A., Grafmiller, J., Rosseel, L., Szmrecsanyi, B., & Van de Velde, F.
    (2021) How register-specific is probabilistic grammatical knowledge?: A programmatic sketch and a case study on the dative alternation with give. InE. Seoane & D. Biber (Eds.), Corpus-based approaches to register variation (pp.–). John Benjamins. 10.1075/scl.103.03eng
     https://doi.org/10.1075/scl.103.03eng [Google Scholar]
  18. Engel, A., & Szmrecsanyi, B.
    (2022) Variable grammars are variable across registers: Future temporal reference in English. Language Variation and Change, (), –. 10.1017/S0954394522000163
     https://doi.org/10.1017/S0954394522000163 [Google Scholar]
  19. Gries, S. T.
    (2003) Towards a corpus-based identification of prototypical instances of constructions. Annual Review of Cognitive Linguistics, (), –. 10.1075/arcl.1.02gri
     https://doi.org/10.1075/arcl.1.02gri [Google Scholar]
  20. Hayes, A. F., & Krippendorff, K.
    (2007) Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, (), –. 10.1080/19312450709336664
     https://doi.org/10.1080/19312450709336664 [Google Scholar]
  21. Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S. and Vespignani, A., & Yarkoni, T.
    (2021) Integrating explanation and prediction in computational social science. Nature, (), –. 10.1038/s41586‑021‑03659‑0
     https://doi.org/10.1038/s41586-021-03659-0 [Google Scholar]
  22. Hovy, D., & Yang, D.
    (2021) The importance of modeling social factors of language: Theory and practice. InK. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, & Y. Zhou (Eds.), Proceedings of the 2021 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies (pp.–). Association for Computational Linguistics. 10.18653/v1/2021.naacl‑main.49
     https://doi.org/10.18653/v1/2021.naacl-main.49 [Google Scholar]
  23. Hu, J., Mahowald, K., Lupyan, G., Ivanova, A., & Levy, R.
    (2024) Language models align with human judgments on key grammatical constructions. Proceedings of the National Academy of Sciences, (), . 10.1073/pnas.2400917121
     https://doi.org/10.1073/pnas.2400917121 [Google Scholar]
  24. Jenset, G. B., McGillivray, B., & Rundell, M.
    (2018) The dative alternation revisited: Fresh insights from contemporary British spoken data. InV. Brezina, R. Love, & K. Aijmer (Eds.), Corpus approaches to contemporary British speech (pp.–). Routledge. 10.4324/9781315268323‑10
     https://doi.org/10.4324/9781315268323-10 [Google Scholar]
  25. Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E.
    (2024) Dissociating language and thought in large language models. Trends in Cognitive Sciences, (), –. 10.1016/j.tics.2024.01.011
     https://doi.org/10.1016/j.tics.2024.01.011 [Google Scholar]
  26. Malmsten, M., Börjeson, L., & Haffenden, C.
    (2020) Playing with words at the National Library of Sweden — Making a Swedish BERT. arXiv, arXiv:2007.. 10.48550/arXiv.2007.01658
     https://doi.org/10.48550/arXiv.2007.01658 [Google Scholar]
  27. Nguyen, D., Rosseel, L., & Grieve, J.
    (2021) On learning and representing social meaning in NLP: A sociolinguistic perspective. InK. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, & Y. Zhou (Eds.), Proceedings of the 2021 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies (pp.–). Association for Computational Linguistics. 10.18653/v1/2021.naacl‑main.50
     https://doi.org/10.18653/v1/2021.naacl-main.50 [Google Scholar]
  28. Peters, M. E., Neumann, M., Zettlemoyer, L., & Yih, W. -T.
    (2018) Dissecting contextual word embeddings: Architecture and representation. InE. Riloff, D. Chiang, J. Hockenmaier, & J. Tsujii (Eds.), Proceedings of the 2018 conference on empirical methods in Natural Language Processing (pp.–). Association for Computational Linguistics. 10.18653/v1/D18‑1179
     https://doi.org/10.18653/v1/D18-1179 [Google Scholar]
  29. Pijpops, D., Franco, K., Speelman, D., & Van de Velde, F.
    (2024) Introduction: What are alternations and how should we study them?Linguistics Vanguard, (), –. 10.1515/lingvan‑2023‑0165
     https://doi.org/10.1515/lingvan-2023-0165 [Google Scholar]
  30. R Core Team
    R Core Team (2022) R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/.
     [Google Scholar]
  31. Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B.
    (2018) Gender bias in coreference resolution. InM. Walker, H. Ji, & A. Stent (Eds.), Proceedings of the 2018 conference of the North American chapter of the Association for Computational Linguistics: Human language technologies, Volume 2 (Short Papers) (pp.–). Association for Computational Linguistics. 10.18653/v1/N18‑2002
     https://doi.org/10.18653/v1/N18-2002 [Google Scholar]
  32. Salazar, J., Liang, D., Nguyen, T. Q., & Kirchhoff, K.
    (2020) Masked language model scoring. InD. Jurafsky, J. Chai, N. Schluter, & J. Tetreault (Eds.), Proceedings of the 58th annual meeting of the Association for Computational Linguistics (pp.–). Association for Computational Linguistics. 10.18653/v1/2020.acl‑main.240
     https://doi.org/10.18653/v1/2020.acl-main.240 [Google Scholar]
  33. Sundararajan, M., Taly, A., & Yan, Q.
    (2017) Axiomatic attribution for deep networks. Proceedings of the 34th International Conference on Machine Learning, , –. https://proceedings.mlr.press/v70/sundararajan17a.html.
     [Google Scholar]
  34. Szmrecsanyi, B., Grafmiller, J., Bresnan, J., Rosenbach, A., Tagliamonte, S., & Todd, S.
    (2017) Spoken syntax in a comparative perspective: The dative and genitive alternation in varieties of English. Glossa: A Journal of General Linguistics, (), . 10.5334/gjgl.310
     https://doi.org/10.5334/gjgl.310 [Google Scholar]
  35. Tagliamonte, S. A., Durham, M. & Smith, J.
    (2014) Grammaticalization at an early stage: Future be going to in conservative British dialects. English Language and Linguistics, (), –. 10.1017/S1360674313000282
     https://doi.org/10.1017/S1360674313000282 [Google Scholar]
  36. Tenney, I., Das, D., & Pavlick, E.
    (2019) BERT Rediscovers the classical NLP pipeline. InA. Korhonen, D. Traum, & L. Màrquez (Eds.), Proceedings of the 57th annual meeting of the Association for Computational Linguistics (pp.–). Association for Computational Linguistics. 10.18653/v1/P19‑1452
     https://doi.org/10.18653/v1/P19-1452 [Google Scholar]
  37. Theijssen, D., Ten Bosch, L., Boves, L., Cranen, B., & Van Halteren, H.
    (2013) Choosing alternatives: Using Bayesian networks and memory-based learning to study the dative alternation. Corpus Linguistics and Linguistic Theory, (), –. 10.1515/cllt‑2013‑0007
     https://doi.org/10.1515/cllt-2013-0007 [Google Scholar]
  38. Van den Bosch, A., & Bresnan, J.
    (2015) Modeling dative alternations of individual children. InR. Berwick, A. Korhonen, A. Lenci, T. Poibeau, & A. Villavicencio (Eds.), Proceedings of the sixth workshop on cognitive aspects of computational language learning (pp.–). Association for Computational Linguistics. 10.18653/v1/W15‑2414
     https://doi.org/10.18653/v1/W15-2414 [Google Scholar]
  39. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I.
    (2017) Attention is all you need. InI. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems, Volume (pp.–). NeurIPS. https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
     [Google Scholar]
  40. Williams, R. S.
    (1994) A statistical analysis of English double object alternation. Issues in Applied Linguistics, (), –. 10.5070/L451005170
     https://doi.org/10.5070/L451005170 [Google Scholar]
  41. Wolk, C., Bresnan, J., Rosenbach, A., & Szmrecsanyi, B.
    (2013) Dative and genitive variability in Late Modern English: Exploring cross-constructional variation and change. Diachronica, (), –. 10.1075/dia.30.3.04wol
     https://doi.org/10.1075/dia.30.3.04wol [Google Scholar]
/content/journals/10.1075/ijcl.24065.mor
Loading
Not all linguistic variation is equally predictable
John Benjamins ; https://doi.org/10.1075/ijcl.24065.mor
/content/journals/10.1075/ijcl.24065.mor
/content/journals/10.1075/ijcl.24065.mor
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: Swedish ; language variation ; prediction ; large language models ; English

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error