1887
  1. Home
  2. e-Journals
  3. Pedagogical Linguistics
  4. Online First Articles
  5. Online First Article
image of Measuring grammatical understanding
PDF
EPUB

Abstract

Abstract

In the realm of first language (L1) grammar education, research on instruments assessing explicit grammatical understanding is limited. This article discusses an Item Response Theory (IRT) analysis on the Test for Grammatical Understanding (TGU), a multiple-choice tool designed for measuring such grammatical understanding, in this case pertaining to the concept of valency and related concepts (e.g., subject, objects). The results draw from 189 pre-university students (aged 14), who completed different versions of the TGU at multiple time points. Using IRT models for categorical data, we assessed item difficulty, discrimination, and response category functioning. Results highlight the TGU’s usability, version interchangeability, and its potential as a tool for monitoring progress in grammatical understanding. Additionally, differential item functioning (DIF) analyses indicated that most test items performed consistently across versions, though a few showed discrepancies requiring further examination. The discussion addresses item success variations, proposing items that might be omitted under time constraints, and examines the value of a ‘partial understanding’ answer category alongside categories reflecting ‘full’ and ‘no understanding.’ Future study recommendations include expanding TGUs to different linguistic concepts beyond valency and exploring its correlation with open reasoning tasks to further validate its effectiveness in assessing explicit grammatical understanding.

© John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/pl.24014.van
2025-03-18
2025-04-25

  • From This Site

    /content/journals/10.1075/pl.24014.van
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/10.1075/pl.24014.van/pl.24014.van.html?itemId=/content/journals/10.1075/pl.24014.van&mimeType=html&fmt=ahah

References

  1. Alderson, J., & Hudson, R.
    (2013) The metalinguistic knowledge of undergraduate students of English language or linguistics. Language Awareness, (), –. 10.1080/09658416.2012.722644
     https://doi.org/10.1080/09658416.2012.722644 [Google Scholar]
  2. Arseneau, R., Foucambert, D., & Lefrançois, P.
    (2018) Improving the mastery of relative clause in French L1 secondary classes: The effects of an intervention based on verbal interactions on written syntactic structures. L1-Educational Studies in Language and Literature, (), –. 10.17239/L1ESLL‑2018.18.01.08
     https://doi.org/10.17239/L1ESLL-2018.18.01.08 [Google Scholar]
  3. Baumberger, C.
    (2019) Explicating objectual understanding: Taking degrees seriously. Journal for General Philosophy of Science, , –. 10.1007/s10838‑019‑09474‑6
     https://doi.org/10.1007/s10838-019-09474-6 [Google Scholar]
  4. Boone, W. J.
    (2016) Rasch analysis for instrument development: Why, when, and how?CBE — Life Sciences Education, (), –. 10.1187/cbe.16‑04‑0148
     https://doi.org/10.1187/cbe.16-04-0148 [Google Scholar]
  5. Boivin, M.-C., Fontich, X., Funke, R., García-Folgado, M.-J., & Myhill, D.
    (2018) Working on grammar at school in L1 education: Empirical research across linguistic regions. L1-Educational Studies in Language and Literature, (), –. 10.17239/L1ESLL‑2018.18.04.01
     https://doi.org/10.17239/L1ESLL-2018.18.04.01 [Google Scholar]
  6. Brøseth, H., & Nygård, M.
    (2023) Norwegian first-year student teachers’ knowledge of L1 grammar. L1-Educational Studies in Language and Literature, (), –. 10.21248/l1esll.2023.23.1.411
     https://doi.org/10.21248/l1esll.2023.23.1.411 [Google Scholar]
  7. Broekhuis, H., Corver, N. & Vos, R.
    (2015) Syntax of Dutch: Verbs and Verb Phrases. Volume 1. Amsterdam University Press.
     [Google Scholar]
  8. Cai, L., & Hansen, M.
    (2013) Limited-information goodness-of-fit testing of hierarchical item factor models. British Journal of Mathematical and Statistical Psychology, , –. 10.1111/j.2044‑8317.2012.02050.x
     https://doi.org/10.1111/j.2044-8317.2012.02050.x [Google Scholar]
  9. Chalmers, R.
    (2012) mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, (), –. 10.18637/jss.v048.i06
     https://doi.org/10.18637/jss.v048.i06 [Google Scholar]
  10. Chen, H., & Myhill, D.
    (2016) Children talking about writing: Investigating metalinguistic understanding. Linguistics and Education, , –. 10.1016/j.linged.2016.07.004
     https://doi.org/10.1016/j.linged.2016.07.004 [Google Scholar]
  11. Choi, S. W., Gibbons, L. E., & Crane, P. K.
    (2016) lordif: Logistic ordinal regression differential item functioning using IRT (Version 0.3–3) [R package]. https://CRAN.R-project.org/package=lordif
  12. Coppen, P.-A., Van Rijt, J., Wijnands, A., & Dielemans, R.
    (2019) Vakdidactisch onderzoek naar grammaticaonderwijs in Nederland. Tijdschrift voor Nederlandse Taal- en Letterkunde, (), –. 10.5117/TNTL2019.2.002.COPP
     https://doi.org/10.5117/TNTL2019.2.002.COPP [Google Scholar]
  13. Cronbach, L. J.
    (1951) Coefficient alpha and the internal structure of tests. Psychometrika, , –. 10.1007/BF02310555
     https://doi.org/10.1007/BF02310555 [Google Scholar]
  14. Dielemans, R., & Coppen, P.-A.
    (2020) Defining linguistic reasoning: Transposing and grounding a model for historical reasoning to the linguistic domain. Dutch Journal of Applied Linguistics, (), –. 10.1075/dujal.19038.die
     https://doi.org/10.1075/dujal.19038.die [Google Scholar]
  15. Fenwick, L., Humphrey, S., Quinn, M., & Endicott, M.
    (2013) Developing deep understanding about language in undergraduate pre-service teacher programs through the application of knowledge. Australian Journal of Teacher Education, (), –. 10.14221/ajte.2014v39n1.4
     https://doi.org/10.14221/ajte.2014v39n1.4 [Google Scholar]
  16. Fontich, X.
    (2014) Grammar and language reflection at school: Checking out the whats and the hows of grammar instruction. InT. Ribas, X. Fontich, & O. Guasch (Eds.), Grammar at school: Research on metalinguistic activity in language education (pp.–). Peter Lang.
     [Google Scholar]
  17. Fontich, X., & Camps, A.
    (2014) Towards a rationale for research into grammar teaching at schools. Research Papers in Education, (), –. 10.1080/02671522.2013.813579
     https://doi.org/10.1080/02671522.2013.813579 [Google Scholar]
  18. Fontich, X., Van Rijt, J., & Gauvin, I.
    (2020) Research on L1 grammar in schooling: Mediation at the heart of learning grammar. L1-Educational Studies in Language and Literature, (), –. 10.17239/L1ESLL‑2020.20.03.01
     https://doi.org/10.17239/L1ESLL-2020.20.03.01 [Google Scholar]
  19. Giovanelli, M.
    (2015) Becoming an English language teacher: Linguistic knowledge, anxieties and the shifting sense of identity. Language and Education, (), –. 10.1080/09500782.2015.1031677
     https://doi.org/10.1080/09500782.2015.1031677 [Google Scholar]
  20. Graham, S., Kim, Y.-S., Cao, Y., Lee, J., Tate, T., Collins, P., Cho, M., Moon, Y., Chung, H. Q., & Olson, C. B.
    (2023) A meta-analysis of writing treatments for students in grades 6–12. Journal of Educational Psychology, (), –. 10.1037/edu0000819
     https://doi.org/10.1037/edu0000819 [Google Scholar]
  21. Hmelo-Silver, C. E., & Pfeffer, M.
    (2004) Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, , –. 10.1207/s15516709cog2801_7
     https://doi.org/10.1207/s15516709cog2801_7 [Google Scholar]
  22. Land, R., Meyer, J. H. F., & Smith, J.
    (2008) Threshold concepts within the disciplines. BRILL. 10.1163/9789460911477
     https://doi.org/10.1163/9789460911477 [Google Scholar]
  23. Luif, J.
    (1994) In verband met de zin: Inleiding in de Nederlandse Spraakkunst (3rd ed.). Amsterdam University Press.
     [Google Scholar]
  24. Instituut voor de Nederlandse Taal
    Instituut voor de Nederlandse Taal (2021) Algemene Nederlandse Spraakkunst. Third, revised version.
     [Google Scholar]
  25. Jones, P., & Chen, H.
    (2012) Teachers’ knowledge about language: Issues of pedagogy and expertise. Australian Journal of Language and Literacy, (), –. 10.1007/BF03651880
     https://doi.org/10.1007/BF03651880 [Google Scholar]
  26. Leenders, G., de Graaff, H., & van Koppen, J. M.
    (2023) Hoe meet je bewuste taalvaardigheid? Grammaticaal redeneren in de vakken Nederlands, Engels en Duits. Pedagogische Studiën, (). 10.51751/dujal16273
     https://doi.org/10.51751/dujal16273 [Google Scholar]
  27. Locke, T.
    (Ed.) (2010) Beyond the grammar wars: A resource for teachers and students on developing language knowledge in the English/literacy classroom. Routledge. 10.4324/9780203854358
     https://doi.org/10.4324/9780203854358 [Google Scholar]
  28. Maydeu-Olivares, A., & Joe, H.
    (2014) Assessing Approximate Fit in Categorical Data Analysis. Multivariate Behavioral Research, (), –. 10.1080/00273171.2014.911075
     https://doi.org/10.1080/00273171.2014.911075 [Google Scholar]
  29. Masters, G. N.
    (1982) A Rasch model for partial credit scoring. Psychometrika, , –. 10.1007/BF02296272
     https://doi.org/10.1007/BF02296272 [Google Scholar]
  30. Muraki, E.
    (1992) A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, , –. 10.1177/014662169201600206
     https://doi.org/10.1177/014662169201600206 [Google Scholar]
  31. Myhill, D.
    (2000) Misconceptions and difficulties in the acquisition of metalinguistic knowledge. Language and Education, (), –. 10.1080/09500780008666787
     https://doi.org/10.1080/09500780008666787 [Google Scholar]
  32. (2003) Principled understanding? Teaching the active and passive voice. Language and Education, (), –. 10.1080/09500780308666856
     https://doi.org/10.1080/09500780308666856 [Google Scholar]
  33. (2018) Grammar as a meaning-making resource for language development. L1-Educational Studies in Language and Literature, (), –. 10.17239/L1ESLL‑2018.18.04.04
     https://doi.org/10.17239/L1ESLL-2018.18.04.04 [Google Scholar]
  34. (2021) Grammar re-imagined: Foregrounding understanding of language choice in writing. English in Education, (), –. 10.1080/04250494.2021.1885975
     https://doi.org/10.1080/04250494.2021.1885975 [Google Scholar]
  35. Myhill, D., Jones, S., & Lines, H.
    (2018) Supporting less proficient writers through linguistically aware teaching. Language and Education, (), –. 10.1080/09500782.2018.1438468
     https://doi.org/10.1080/09500782.2018.1438468 [Google Scholar]
  36. Myhill, D., Jones, S., & Watson, A.
    (2013) Grammar matters: How teachers’ grammatical knowledge impacts on the teaching of writing. Teaching and Teacher Education, , –. 10.1016/j.tate.2013.07.005
     https://doi.org/10.1016/j.tate.2013.07.005 [Google Scholar]
  37. Perini, M.
    (2015) Describing verb valency: Practical and theoretical issues. Springer. 10.1007/978‑3‑319‑20985‑2
     https://doi.org/10.1007/978-3-319-20985-2 [Google Scholar]
  38. Rasch, G.
    (1960) Probabilistic models for some intelligence and achievement tests. Danish Institute for Educational Research.
     [Google Scholar]
  39. Rättyä, K., Awramiuk, E., & Fontich, X.
    (2019) What is grammar in L1 education today?L1-Educational Studies in Language and Literature, (), –. 10.17239/L1ESLL‑2019.19.02.01
     https://doi.org/10.17239/L1ESLL-2019.19.02.01 [Google Scholar]
  40. Samejima, F.
    (1969) Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, . 10.1007/BF03372160
     https://doi.org/10.1007/BF03372160 [Google Scholar]
  41. Sangster, P., Anderson, C., & O’Hara, P.
    (2013) Perceived and actual levels of knowledge about language amongst primary and secondary student teachers: Do they know what they think they know?Language Awareness, (), –. 10.1080/09658416.2012.722643
     https://doi.org/10.1080/09658416.2012.722643 [Google Scholar]
  42. Taber, K. S.
    (2018) The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, , –. 10.1007/s11165‑016‑9602‑2
     https://doi.org/10.1007/s11165-016-9602-2 [Google Scholar]
  43. Trotzke, A.
    (2023) Pedagogical linguistics: Connecting formal linguistics to language teaching. Language, (), –. 10.1353/lan.2023.a907016
     https://doi.org/10.1353/lan.2023.a907016 [Google Scholar]
  44. Van Gelderen, A.
    (2010) Does explicit teaching of grammar help students to become better writers? Insights from empirical research. InT. Locke (Ed.), Beyond the grammar wars: A resource for teachers and students on developing language knowledge in the English/literacy classroom (pp.–). Routledge.
     [Google Scholar]
  45. Stadler, M., Sailer, M., & Fischer, F.
    (2021) Knowledge as a formative construct: A good alpha is not always better. New Ideas in Psychology, . 10.1016/j.newideapsych.2020.100832
     https://doi.org/10.1016/j.newideapsych.2020.100832 [Google Scholar]
  46. Van Rijt, J.
    (2020) Understanding grammar: the impact of linguistic metaconcepts on L1 grammar education. Doctoral dissertation. Radboud University Nijmegen.
  47. (2024) Learning how to think like a linguist: Linguistic reasoning as a focal point in L1 grammar education. Language and Linguistics Compass, (), –. Article e12513. 10.1111/lnc3.12513
     https://doi.org/10.1111/lnc3.12513 [Google Scholar]
  48. Van Rijt, J., & Coppen, P.-A. J. M.
    (2021) The conceptual importance of grammar: Knowledge-related rationales for grammar teaching. Pedagogical Linguistics, (), –. https://benjamins.com/catalog/pl.21008.van. 10.1075/pl.21008.van
     https://doi.org/10.1075/pl.21008.van [Google Scholar]
  49. Van Rijt, J., Banga, A., & Goudbeek, M.
    (2024) Getting a load of linguistic reasoning: How L1 student teachers process rules of thumb and linguistic manipulations in discussions about grammar. Applied Linguistics, (), –. 10.1093/applin/amad011
     https://doi.org/10.1093/applin/amad011 [Google Scholar]
  50. Van Rijt, J. H. M., de Swart, P., Wijnands, A., & Coppen, P.-A.
    (2019) When students tackle grammatical problems: Exploring linguistic reasoning with linguistic metaconcepts in L1 grammar education. Linguistics and Education, , –. 10.1016/j.linged.2019.06.004
     https://doi.org/10.1016/j.linged.2019.06.004 [Google Scholar]
  51. Van Rijt, J., Hulshof, H., & Coppen, P.-A.
    (2021) ‘X is the odd one out, because the other two are more about the farmland’: Dutch L1 student teachers’ struggles to reason about grammar in odd one out tasks. Studies in Educational Evaluation, , Article 101007. 10.1016/j.stueduc.2021.101007
     https://doi.org/10.1016/j.stueduc.2021.101007 [Google Scholar]
  52. Van Rijt, J., Myhill, D., De Maeyer, S., & Coppen, P.-A.
    (2022) Linguistic metaconcepts can improve grammatical understanding in L1 education evidence from a Dutch quasi-experimental study. PLoS One, (), Article e0263123. 10.1371/journal.pone.0263123
     https://doi.org/10.1371/journal.pone.0263123 [Google Scholar]
  53. Van Rijt, J., Wijnands, A., & Coppen, P.-A.
    (2020) How secondary school students may benefit from linguistic metaconcepts to reason about L1 grammatical problems. Language and Education, (), –. 10.1080/09500782.2019.1690503
     https://doi.org/10.1080/09500782.2019.1690503 [Google Scholar]
  54. Watson, A., & Newman, R.
    (2017) Talking grammatically: L1 adolescent metalinguistic reflection on writing. Language Awareness, (), –. 10.1080/09658416.2017.1410554
     https://doi.org/10.1080/09658416.2017.1410554 [Google Scholar]
  55. Wijnands, A., van Rijt, J., Stoel, G., & Coppen, P.-A.
    (2022) Balancing between uncertainty and control: Teaching reflective thinking about language in the classroom. Linguistics and Education, , –. 10.1016/j.linged.2022.101087
     https://doi.org/10.1016/j.linged.2022.101087 [Google Scholar]
  56. Xu, J., Paek, I., & Xia, Y.
    (2017) Investigating the Behaviors of M2 and RMSEA2 in Fitting a Unidimensional Model to Multidimensional Data. Applied Psychological Measurement, (), –. 10.1177/0146621617710464
     https://doi.org/10.1177/0146621617710464 [Google Scholar]
/content/journals/10.1075/pl.24014.van
Loading
Measuring grammatical understanding
John Benjamins ; https://doi.org/10.1075/pl.24014.van
/content/journals/10.1075/pl.24014.van
/content/journals/10.1075/pl.24014.van
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: grammatical reasoning ; IRT analyses ; grammar testing ; TGU ; L1 grammar ; grammatical understanding

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error