1887
  1. Home
  2. e-Journals
  3. ITL - International Journal of Applied Linguistics
  4. Volume 174, Issue 1
  5. Article
Volume 174, Issue 1
  • ISSN 0019-0829
  • E-ISSN: 1783-1490
USD
Buy:$35.00 + Taxes

Abstract

This study presents the development and validation of a 132-item Spanish-English bilingual multiple-choice vocabulary test based on the 3,000 most frequent lemmas that distinguishes between North American university students who satisfy the Foreign Language requirement and those who need to complete coursework. 819 students were assigned to one of the two 144-item forms of the preliminary test, which had 72 shared anchor items and other 72 form-specific items. Factor analysis was used to evaluate dimensionality and the Rasch model was used to select the items that best differentiated between these two student populations. This final form was administered to 213 students. Results showed high levels of unidimensionality, and the final form provided a Rasch reliability coefficient of 0.97.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/itl.22008.rob
2022-10-25
2026-05-21

  • From This Site

    /content/journals/10.1075/itl.22008.rob
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L.
    (2013) Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(6), 1–13. 10.7275/qv2q‑rk76
     https://doi.org/10.7275/qv2q-rk76 [Google Scholar]
  2. Beglar, D.
    (2010) A Rasch-based validation of the vocabulary size test. Language Testing, 27(1), 101–118. 10.1177/0265532209340194
     https://doi.org/10.1177/0265532209340194 [Google Scholar]
  3. Berendt, E., Kurosaki, E., Maeda, A., Matsui, K., & Ochi, N.
    (2006) English loan words of Japanese elementary school children and their mental lexicon. Asian Englishes, 8(2), 26–45. 10.1080/13488678.2005.10801165
     https://doi.org/10.1080/13488678.2005.10801165 [Google Scholar]
  4. Biemiller, A.
    (2003) Vocabulary: Needed if more children are to read well. Reading Psychology, 24(3–4), 323–335. 10.1080/02702710390227297
     https://doi.org/10.1080/02702710390227297 [Google Scholar]
  5. Bond, T. G., & Fox, C. M.
    (2015) Applying the Rasch model: Fundamental measurement in the human sciences. Routledge. 10.4324/9781315814698
     https://doi.org/10.4324/9781315814698 [Google Scholar]
  6. Bürkner, P. C.
    (2017) brms: An R package for bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. 10.18637/jss.v080.i01
     https://doi.org/10.18637/jss.v080.i01 [Google Scholar]
  7. Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddel, A.
    (2017) Stan: A probabilistic programming language. Journal of Statistical Software, 76(1), 1–32. 10.18637/jss.v076.i01
     https://doi.org/10.18637/jss.v076.i01 [Google Scholar]
  8. Casado, M. A., & Dereshiwsky, M. I.
    (2001) Foreign language anxiety of university students. College Student Journal, 35(4), 539–552.
     [Google Scholar]
  9. Chalmers, R. P.
    (2012) mirt: A Multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. 10.18637/jss.v048.i06
     https://doi.org/10.18637/jss.v048.i06 [Google Scholar]
  10. Davies, M.
    (2005) Vocabulary range and text coverage: Insights from the forthcoming Routledge frequency dictionary of Spanish. InD. Eddington (Ed.), Selected proceedings of the 7th Hispanic Linguistics Symposium (pp.106–115). Somerville, MA: Cascadilla Proceedings Project.
     [Google Scholar]
  11. Davies, M., & Hayward Davies, K.
    (2017) A frequency dictionary of Spanish: Core vocabulary for learners (2nd ed.). Routledge. 10.4324/9781315542638
     https://doi.org/10.4324/9781315542638 [Google Scholar]
  12. de Ayala, R. J.
    (2009) The theory and practice of item response theory. The Guilford Press.
     [Google Scholar]
  13. De Boeck, P., & Wilson, M.
    (2004) Explanatory item response models: A generalized linear and nonlinear approach. Springer. 10.1007/978‑1‑4757‑3990‑9
     https://doi.org/10.1007/978-1-4757-3990-9 [Google Scholar]
  14. Desjardins, C. D., & Bulut, O.
    (2018) Handbook of educational measurement and psychometrics using R. CRC Press. 10.1201/b20498
     https://doi.org/10.1201/b20498 [Google Scholar]
  15. Dorans, N. J., Moses, T. P., & Eignor, D. R.
    (2010) Principles and practices of test score equating. ETS Research Report Series 2010(2), i–41. 10.1002/j.2333‑8504.2010.tb02236.x
     https://doi.org/10.1002/j.2333-8504.2010.tb02236.x [Google Scholar]
  16. Frantzen, D., & Magnan, S. S.
    (2005) Anxiety and the true beginner – false beginner dynamic in beginning French and Spanish classes. Foreign Language Annals, 38(2), 171–186. 10.1111/j.1944‑9720.2005.tb02483.x
     https://doi.org/10.1111/j.1944-9720.2005.tb02483.x [Google Scholar]
  17. Floyd, F. J., & Widaman, K. F.
    (1995) Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. 10.1037/1040‑3590.7.3.286
     https://doi.org/10.1037/1040-3590.7.3.286 [Google Scholar]
  18. Gairns, R., & Redman, S.
    (1986) Working with words. A guide to teaching and learning vocabulary. Cambridge University Press.
     [Google Scholar]
  19. Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B.
    (2013) Bayesian data analysis (3rd ed.). CRC Press. 10.1201/b16018
     https://doi.org/10.1201/b16018 [Google Scholar]
  20. Gyllstad, H., Vilkaitė, L., & Schmitt, N.
    (2015) Assessing vocabulary size through multiple-choice formats: Issues with guessing and sampling rates. International Journal of Applied Linguistics, 166(2), 278–306. 10.1075/itl.166.2.04gyl
     https://doi.org/10.1075/itl.166.2.04gyl [Google Scholar]
  21. Hambleton, R. K., Swaminathan, H., & Rogers, H. J.
    (1991) Fundamentals of item response theory. SAGE Publications, Inc.
     [Google Scholar]
  22. Horn, J. L.
    (1965) A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. 10.1007/BF02289447
     https://doi.org/10.1007/BF02289447 [Google Scholar]
  23. Horwitz, E.
    (1990) Attending to the affective domain in the foreign language classroom. InS. Magnan (Ed.), Shifting the instructional focus to the learner (pp.15–33). Northeast Conference on the Teaching of Foreign Languages.
     [Google Scholar]
  24. Hu, M., & Nation, I. S. P.
    (2000) Vocabulary density and reading comprehension. Reading in a Foreign Language, 23(1), 4031–430.
     [Google Scholar]
  25. Institute for Test Research and Test Development
    Institute for Test Research and Test Development (2013) Assessing evidence of validity of the ACTFL reading proficiency test (RPT).
     [Google Scholar]
  26. Izura, C., Cuetos, F., & Brysbaert, M.
    (2014) Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35(1), 49–66.
     [Google Scholar]
  27. Jones, R., & Tschirner, E.
    (2006) A frequency dictionary of German: Core vocabulary for learners. Routledge.
     [Google Scholar]
  28. Kaplan, R. M., & Saccuzzo, D. P.
    (2009) Psychological testing principles, applications, and issues (7th ed.). Wadsworth.
     [Google Scholar]
  29. Kruschke, J. K.
    (2015) Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Elsevier.
     [Google Scholar]
  30. Laufer, B., & Aviad-Levitzky, A.
    (2017) What type of vocabulary knowledge predicts reading comprehension: Word meaning recall or word meaning recognition?The Modern Language Journal, 101(4), 729–741. https://onlinelibrary.wiley.com/doi/abs/10.1111/modl.12431
     [Google Scholar]
  31. Laufer, B., & McLean, S.
    (2016) Loanwords and vocabulary size test scores: A case of different estimates for different L1 learners. Language Assessment Quarterly, 13(3), 202–217. 10.1080/15434303.2016.1210611
     https://doi.org/10.1080/15434303.2016.1210611 [Google Scholar]
  32. Liu, N., & Nation, I. S. P.
    (1985) Factors affecting guessing vocabulary in context. RELC Journal, 16(1), 33–42. 10.1177/003368828501600103
     https://doi.org/10.1177/003368828501600103 [Google Scholar]
  33. Looney, D., & Lusin, N.
    (2019) Enrollments in languages other than English in United States institutions of higher education, summer 2016 and fall 2016: Final report. Modern Language Association of America. https://www.mla.org/content/download/110154/2406932/2016-Enrollments-Final-Report.pdf
     [Google Scholar]
  34. McElreath, R.
    (2020) Statistical rethinking (2nd ed.). CRC Press. 10.1201/9780429029608
     https://doi.org/10.1201/9780429029608 [Google Scholar]
  35. McLean, S., Kramer, B., & Beglar, D.
    (2015a) The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741–760. 10.1177/1362168814567889
     https://doi.org/10.1177/1362168814567889 [Google Scholar]
  36. McLean, S., Kramer, B., & Stewart, J.
    (2015b) An empirical examination of the effect of guessing on vocabulary size test scores. Vocabulary Learning and Instruction, 4(1), 26–35. 10.7820/vli.v04.1.mclean.et.al
     https://doi.org/10.7820/vli.v04.1.mclean.et.al [Google Scholar]
  37. McLean, S., Stewart, J., & Batty, A. O.
    (2020) Predicting L2 reading proficiency with modalities of vocabulary knowledge: A bootstrapping approach. Language Testing, 37(3), 389–411. 10.1177/0265532219898380
     https://doi.org/10.1177/0265532219898380 [Google Scholar]
  38. McNamara, T., & Knoch, U.
    (2012) The Rasch wars: The emergence of Rasch measurement in language testing. Language Testing, 29(4), 555–576. 10.1177/0265532211430367
     https://doi.org/10.1177/0265532211430367 [Google Scholar]
  39. Meara, P.
    (2010) EFL vocabulary test (2nd ed.). Centre for Applied Language Studies.
     [Google Scholar]
  40. Meara, P., & Milton, J.
    (2003) The Swansea levels test. Express.
     [Google Scholar]
  41. Milton, J.
    (2009) Measuring second language vocabulary acquisition. Multilingual Matters. 10.21832/9781847692092
     https://doi.org/10.21832/9781847692092 [Google Scholar]
  42. Nation, I. S. P.
    (2006) How large a vocabulary is needed for reading and listening?Canadian Modern Language Review, 63(1), 591–82. 10.3138/cmlr.63.1.59
     https://doi.org/10.3138/cmlr.63.1.59 [Google Scholar]
  43. Nation, I. S. P., & Beglar, D.
    (2007) A vocabulary size test. The Language Teacher, 31(7), 96–13.
     [Google Scholar]
  44. Nation, I. S. P., & Webb, S. A.
    (2011) Researching and analyzing vocabulary. Cengage.
     [Google Scholar]
  45. Nguyen, L. T. C., & Nation, I. S. P.
    (2011) A bilingual vocabulary size test of English for Vietnamese learners. RELC Journal, 42(1), 86–99. 10.1177/0033688210390264
     https://doi.org/10.1177/0033688210390264 [Google Scholar]
  46. O’Connor, B. P.
    (2000) SPSS and BAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402. 10.3758/BF03200807
     https://doi.org/10.3758/BF03200807 [Google Scholar]
  47. Pellicer-Sánchez, A., & Schmitt, N.
    (2012) Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language Testing, 29(4), 489–509. 10.1177/0265532212438053
     https://doi.org/10.1177/0265532212438053 [Google Scholar]
  48. R Core Team
    R Core Team (2019) R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
     [Google Scholar]
  49. Revelle, W.
    (2020) Psych: Procedures for psychological, psychometric, and personality research. Northwestern University. https://CRAN.R-project.org/package=psych
     [Google Scholar]
  50. Robles-García, P.
    (2020) 3K-LEx: Desarrollo y validación de una prueba de amplitud Léxica en español. Journal of Spanish Language Teaching, 7(1), 64–76. 10.1080/23247797.2020.1770465
     https://doi.org/10.1080/23247797.2020.1770465 [Google Scholar]
  51. (2022) Receptive vocabulary knowledge in L2 learners of Spanish: The role of high-frequency words. Foreign Language Annals, 1–22. 10.1111/flan.12630
     https://doi.org/10.1111/flan.12630 [Google Scholar]
  52. Schmith, R.
    (2002) Qualtrics [survey software]. Utah, U.S.available athttps://www.qualtrics.com
  53. Schmitt, N., Cobb, T., Horst, M., & Schmitt, D.
    (2017) How much vocabulary is needed to use English?Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212–226. 10.1017/S0261444815000075
     https://doi.org/10.1017/S0261444815000075 [Google Scholar]
  54. Schmitt, N., Jiang, X., & Grabe, W.
    (2011) The percentage of words known in a text and reading comprehension. Modern Language Journal, 95(1), 26–43. 10.1111/j.1540‑4781.2011.01146.x
     https://doi.org/10.1111/j.1540-4781.2011.01146.x [Google Scholar]
  55. Schmitt, N. & Schmitt, D.
    (2014) A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484–503. 10.1017/S0261444812000018
     https://doi.org/10.1017/S0261444812000018 [Google Scholar]
  56. Schmitt, N., Schmitt, D., & Clapham, C.
    (2001) Developing and exploring the behaviour of two new versions of the vocabulary levels test. Language Testing, 18(1), 55–88. 10.1177/026553220101800103
     https://doi.org/10.1177/026553220101800103 [Google Scholar]
  57. Stoeckel, T.
    (2018) High-frequency and academic English vocabulary growth among first-year students at UNP. Journal of International Studies and Regional Development, 91, 15–30.
     [Google Scholar]
  58. Stoeckel, T., McLean, S., & Nation, I. S. P.
    (2021) Limitations of size and levels tests of written receptive vocabulary knowledge. Studies in Second Language Acquisition, 43(1), 181–203. 10.1017/S027226312000025X
     https://doi.org/10.1017/S027226312000025X [Google Scholar]
  59. Van Zeeland, H., & Schmitt, N.
    (2012) Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension?Applied Linguistics, 34(4), 457–479. 10.1093/applin/ams074
     https://doi.org/10.1093/applin/ams074 [Google Scholar]
  60. Webb, S. A., & Chang, A. C.-S.
    (2012) Second language vocabulary growth. RELC Journal, 43(1), 113–126. 10.1177/0033688212439367
     https://doi.org/10.1177/0033688212439367 [Google Scholar]
  61. Webb, S., Sasao, Y., & Ballance, O.
    (2017) The updated vocabulary levels test: Developing and validating two new forms of the VLT. International Journal of Applied Linguistics, 168(1), 34–70. 10.1075/itl.168.1.02web
     https://doi.org/10.1075/itl.168.1.02web [Google Scholar]
  62. Wilson, M., De Boeck, P., & Carstensen, C. H.
    (2008) Explanatory item response models: A brief introduction. InM. Wilson & P. De Boeck (Eds.), Explanatory Item Response Models (2nd ed.). Springer.
     [Google Scholar]
  63. Zhang, S., & Zhang, X.
    (2020) The relationship between vocabulary knowledge and L2 reading/listening comprehension: A meta-analysis. Language Teaching Research, 26(4), 696–725. 10.1177/1362168820913998
     https://doi.org/10.1177/1362168820913998 [Google Scholar]
  64. Zhang, X.
    (2013) The I don’t know option in the vocabulary size test. TESOL Quarterly, 47(4), 790–811. 10.1002/tesq.98
     https://doi.org/10.1002/tesq.98 [Google Scholar]
  65. Zipf, G.
    (1935) The psychobiology of language: An introduction to dynamic philology. MIT Press.
     [Google Scholar]
/content/journals/10.1075/itl.22008.rob
Loading
3K-LEx-MC
ITL - International Journal of Applied Linguistics 174, 135 (2023); https://doi.org/10.1075/itl.22008.rob
/content/journals/10.1075/itl.22008.rob
/content/journals/10.1075/itl.22008.rob
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): lexical frequency; multiple-choice; Rasch model; Spanish; vocabulary test

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error