1887
  1. Home
  2. e-Journals
  3. ITL - International Journal of Applied Linguistics
  4. Volume 170, Issue 1
  5. Article
Volume 170, Issue 1
  • ISSN 0019-0829
  • E-ISSN: 1783-1490
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study analyzed the contribution of lexical factors to native-speaking raters’ assessments of comprehensibility and nativeness in second language (L2) speech. Using transcribed samples to reduce non-lexical sources of bias, 10 naïve L1 English raters evaluated speech samples from 97 L2 English learners across two tasks (picture description and TOEFL integrated). Subsequently, the 194 transcripts were analyzed through statistical software (e.g., Coh-metrix, VocabProfile) for 29 variables spanning various lexical dimensions. For the picture description task, separation in lexical correlates of the two constructs was found, with distinct lexical measures tied to comprehensibility and nativeness. In the TOEFL integrated task, comprehensibility and nativeness were largely indistinguishable, with identical sets of lexical variables, covering dimensions of diversity and range. Findings are discussed in relation to the acquisition, assessment, and teaching of lexical properties in L2 speech.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/itl.17026.app
2019-04-05
2025-02-14

  • From This Site

    /content/journals/10.1075/itl.17026.app
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Abercrombie, D.
    (1949) Teaching pronunciation. ELT Journal, 3, 113–122. 10.1093/elt/III.5.113
     https://doi.org/10.1093/elt/III.5.113 [Google Scholar]
  2. Adams, M. L.
    (1980) Five co-occurring factors in speaking proficiency. InJ. R. Firth (Ed.), Measuring spoken language proficiency (pp.1–6). Washington, DC: Georgetown University Press.
     [Google Scholar]
  3. Anderson, R. C., & Freebody, P.
    (1981) Vocabulary knowledge. InJ. T. Gutherie (Ed.) Comprehension and teaching: Researching reviews (pp.77–117). Newark, DE. International Reading Association.
     [Google Scholar]
  4. Baayen, R. H., Piepenbrock, R., & Gulikers, L.
    (1995) CELEX. Philadelphia, PA: Linguistic Data Consortium.
     [Google Scholar]
  5. Browne, C., Culligan, B., & Phillips, J.
    (2013) The new general service list. Retrieved from www.newgeneralservicelist.org
  6. Cobb, T.
    (2016) Compleat Lexical Tutor [computer program]. www.lextutor.ca (15January 2016).
     [Google Scholar]
  7. Crossley, S. A., Cobb, T., & McNamara, D.
    (2013) Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System, 41, 965–981. 10.1016/j.system.2013.08.002
     https://doi.org/10.1016/j.system.2013.08.002 [Google Scholar]
  8. Crossley, S. A., & McNamara, D.
    (2013) Applications of text analysis tools for spoken response grading. Language Learning & Technology, 17, 171–192.
     [Google Scholar]
  9. Crossley, S. A., Salsbury, T., & McNamara, D. S.
    (2009) Measuring L2 lexical growth using hypernymic relationships. Language Learning, 59, 307–334. 10.1111/j.1467‑9922.2009.00508.x
     https://doi.org/10.1111/j.1467-9922.2009.00508.x [Google Scholar]
  10. (2010) The development of polysemy and frequency use in English second language speakers. Language Learning, 60, 573–605. 10.1111/j.1467‑9922.2010.00568.x
     https://doi.org/10.1111/j.1467-9922.2010.00568.x [Google Scholar]
  11. (2015) Assessing lexical proficiency using analytic ratings: A case for collocation accuracy. Applied Linguistics, 36, 570–590.
     [Google Scholar]
  12. Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S.
    (2011) What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly, 45, 182–193. 10.5054/tq.2010.244019
     https://doi.org/10.5054/tq.2010.244019 [Google Scholar]
  13. Crossley, S. A., Yang, H. S., & McNamara, D. S.
    (2014) What’s so simple about simplified texts? A computational and psycholinguistic investigation of text comprehension and text processing. Reading in a Foreign Language, 26, 92–113.
     [Google Scholar]
  14. Crowther, D., Trofimovich, P., Isaacs, T., & Saito, K.
    (2015) Does a speaking task affect second language comprehensibility?The Modern Language Journal, 99, 80–95. 10.1111/modl.12185
     https://doi.org/10.1111/modl.12185 [Google Scholar]
  15. Derwing, T. M., & Munro, M. J.
    (2015) Pronunciation fundamentals: Evidence-based perspectives for L2 teaching and research. Amsterdam: John Benjamins. 10.1075/lllt.42
     https://doi.org/10.1075/lllt.42 [Google Scholar]
  16. (2013) The development of L2 oral language skills in two L1 groups: A seven-year study. Language Learning, 63, 163–185. 10.1111/lang.12000
     https://doi.org/10.1111/lang.12000 [Google Scholar]
  17. Derwing, T. M., Rossiter, M. J., & Ehrensberger-Dow, M.
    (2002) They speaked and wrote real good: Judgements of non-native and native grammar. Language Awareness, 11, 84–99. 10.1080/09658410208667048
     https://doi.org/10.1080/09658410208667048 [Google Scholar]
  18. Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I.
    (2004) L2 fluency: Judgements on different tasks. Language Learning, 54, 655–679. 10.1111/j.1467‑9922.2004.00282.x
     https://doi.org/10.1111/j.1467-9922.2004.00282.x [Google Scholar]
  19. Educational Testing Services
    Educational Testing Services (2004) Independent Speaking Scoring Rubrics. Retrieved from www.ets.org/Media/Tests/TOEFL/pdf/Speaking_Rubrics.pdf
  20. Field, A.
    (2009) Discovering statistics using SPSS (3rd ed.). London: Sage.
     [Google Scholar]
  21. Ginther, A., Dimova, S., & Yang, R.
    (2010) Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing, 27, 379–399. 10.1177/0265532210364407
     https://doi.org/10.1177/0265532210364407 [Google Scholar]
  22. Graesser, A. C., McNamara, D. S., & Kulikowich, J. M.
    (2011) Coh-Metrix providing multilevel analyses of text characteristics. Educational Researcher, 40, 223–234. 10.3102/0013189X11413260
     https://doi.org/10.3102/0013189X11413260 [Google Scholar]
  23. Granena, G.
    (2014) Individual differences in sequence learning ability and second language acquisition in early childhood and adulthood. Language Learning, 63, 665–703. 10.1111/lang.12018
     https://doi.org/10.1111/lang.12018 [Google Scholar]
  24. Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S.
    (2008) Assessed levels of second language speaking proficiency: How distinct?Applied Linguistics, 29, 24–49. 10.1093/applin/amm017
     https://doi.org/10.1093/applin/amm017 [Google Scholar]
  25. Jarvis, S., & Daller, M.
    (2013) Vocabulary knowledge: Human ratings and automated measures. Amsterdam: John Benjamins. 10.1075/sibil.47
     https://doi.org/10.1075/sibil.47 [Google Scholar]
  26. Kang, O., Rubin, D., Pickering, L.
    (2010) Suprasegmental measures of accentedness and judgements of English language learner proficiency in oral English. The Modern Language Journal, 94, 554–566. 10.1111/j.1540‑4781.2010.01091.x
     https://doi.org/10.1111/j.1540-4781.2010.01091.x [Google Scholar]
  27. Koizumi, R., & In’nami, Y.
    (2012) Effects of text length on lexical diversity measures: Using short texts with less than 200 tokens. System, 40, 554–564. 10.1016/j.system.2012.10.012
     https://doi.org/10.1016/j.system.2012.10.012 [Google Scholar]
  28. Kormos, J., & Dénes, M.
    (2004) Exploring measures and perceptions of fluency in the speech of second language learners. System, 32, 145–164. 10.1016/j.system.2004.01.001
     https://doi.org/10.1016/j.system.2004.01.001 [Google Scholar]
  29. Kuiken, F., & Vedder, I.
    (2014) Raters’ decisions, rating procedures and rating scales. Language Testing, 31, 279–284. 10.1177/0265532214526179
     https://doi.org/10.1177/0265532214526179 [Google Scholar]
  30. Kyle, K., & Crossley, S.
    (2015) Automatically assessing lexical sophistication: Indices, tools, findings, and applications. TESOL Quarterly, 49, 757–786. 10.1002/tesq.194
     https://doi.org/10.1002/tesq.194 [Google Scholar]
  31. Larson-Hall, J.
    (2010) A guide to doing statistics in second language research using SPSS. New York: Routledge.
     [Google Scholar]
  32. Levis, J.
    (2005) Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 39, 369–377. 10.2307/3588485
     https://doi.org/10.2307/3588485 [Google Scholar]
  33. Lu, X.
    (2012) The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Review, 96, 190–208. 10.1111/j.1540‑4781.2011.01232_1.x
     https://doi.org/10.1111/j.1540-4781.2011.01232_1.x [Google Scholar]
  34. McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z.
    (2014) Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press. 10.1017/CBO9780511894664
     https://doi.org/10.1017/CBO9780511894664 [Google Scholar]
  35. McNamara, T. F.
    (1990) Item response theory and the validation of an ESP test for health professionals. Language Testing, 7, 52–75. 10.1177/026553229000700105
     https://doi.org/10.1177/026553229000700105 [Google Scholar]
  36. Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., & Miller, K. J.
    (1990) Introduction to WordNet: An on-line lexical database. International Journal of Lexicography, 3, 235–244. 10.1093/ijl/3.4.235
     https://doi.org/10.1093/ijl/3.4.235 [Google Scholar]
  37. Moyer, A.
    (2013) Foreign accent: The phenomenon of non-native speech. Cambridge: Cambridge University Press. 10.1017/CBO9780511794407
     https://doi.org/10.1017/CBO9780511794407 [Google Scholar]
  38. Munro, M. J., & Derwing, T. M.
    (2006) The functional load principle in ESL pronunciation instruction: An exploratory study. System, 34, 520–531. 10.1016/j.system.2006.09.004
     https://doi.org/10.1016/j.system.2006.09.004 [Google Scholar]
  39. (2009) Comprehensibility as a factor in listener interaction preferences: Implications for the workplace. The Canadian Modern Language Review, 66, 181–202. 10.3138/cmlr.66.2.181
     https://doi.org/10.3138/cmlr.66.2.181 [Google Scholar]
  40. Munro, M. J., & Derwing, T. M., Burgess, C. S.
    (2010) Detection of nonnative speaker status from content-masked speech. Speech Communication, 52, 626–637. 10.1016/j.specom.2010.02.013
     https://doi.org/10.1016/j.specom.2010.02.013 [Google Scholar]
  41. Nation, I. S. P.
    (2012) The BNC/COCA word family lists (17 September 2012). Unpublished paper. Available at www.victoria.ac.nz/lals/about/staff/paul-nation
  42. Neufeld, S., & Billuroğlu, A.
    (2005) In search of the critical lexical mass: How ‘general’ is the GSL? How ‘academic’ is the AWL?Available at www.academia.edu/download/2951027/8985atj340yff5z.pdf
  43. Oppenheimer, D. M.
    (2008) The secret life of fluency. Trends in Cognitive Sciences, 12, 237–241. 10.1016/j.tics.2008.02.014
     https://doi.org/10.1016/j.tics.2008.02.014 [Google Scholar]
  44. Perfetti, C. A., Landi, N., & Oakhill, J.
    The acquisition of reading comprehension skill. InM. J. Snowling & C. Hulme Eds. The science of reading: A handbook (pp.227–247). Oxford: Blackwell.
     [Google Scholar]
  45. Plonsky, L., & Oswald, F. L.
    (2014) How big is “big”? Interpreting effects sizes in L2 research. Language Learning, 64, 878–912. 10.1111/lang.12079
     https://doi.org/10.1111/lang.12079 [Google Scholar]
  46. Read, J.
    (2004) Plumbing the depths. InP. Bogaards & B. Laufer (Eds.), Vocabulary in second language (pp.209–227). Amsterdam: John Benjamins. 10.1075/lllt.10.15rea
     https://doi.org/10.1075/lllt.10.15rea [Google Scholar]
  47. Révész, A., Michel, M., & Gilabert, R.
    (2015) Measuring cognitive task demands using dual-task methodology, subjective self-ratings, and expert judgements. Studies in Second Language Acquisition, 38(4), 703–737. 10.1017/S0272263115000339
     https://doi.org/10.1017/S0272263115000339 [Google Scholar]
  48. Robinson, P.
    (2005) Cognitive complexity and task sequencing: Studies in a componential framework for second language task design. International Review of Applied Linguistics in Language Teaching, 43, 1–32. 10.1515/iral.2005.43.1.1
     https://doi.org/10.1515/iral.2005.43.1.1 [Google Scholar]
  49. Saito, K., Trofimovich, P., Isaacs, T.
    (2016) Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels. Applied Psycholinguistics, 37, 217–240. 10.1017/S0142716414000502
     https://doi.org/10.1017/S0142716414000502 [Google Scholar]
  50. Saito, K., Webb, S., Trofimovich, P., & Isaacs, T.
    (2015) Lexical profiles of comprehensible second language speech. Studies in Second Language Acquisition, 38, 677–701. 10.1017/S0272263115000297
     https://doi.org/10.1017/S0272263115000297 [Google Scholar]
  51. Salsbury, T., Crossley, S. A., & McNamara, D. S.
    (2011) Psycholinguistic word information in second language oral discourse. Second Language Research, 27, 343–360. 10.1177/0267658310395851
     https://doi.org/10.1177/0267658310395851 [Google Scholar]
  52. Trofimovich, P., & Isaacs, T.
    (2012) Disentangling accent from comprehensibility. Bilingualism: Language and Cognition, 15, 905–916. 10.1017/S1366728912000168
     https://doi.org/10.1017/S1366728912000168 [Google Scholar]
  53. Wen, Q., Wang, L., & Liang, M.
    (2005) Spoken and written English corpus of Chinese learners. Beijing: Foreign Language Teaching and Research Press.
     [Google Scholar]
  54. Wilson, M. D.
    (1988) The MRC psycholinguistic database: Machine readable dictionary. Behavioural Research Methods, Instruments and Computers, 20, 6–11. 10.3758/BF03202594
     https://doi.org/10.3758/BF03202594 [Google Scholar]
  55. Wray, A.
    (2002) Formulaic language and the lexicon. New York, NY: Cambridge University Press. 10.1017/CBO9780511519772
     https://doi.org/10.1017/CBO9780511519772 [Google Scholar]
/content/journals/10.1075/itl.17026.app
Loading
Lexical aspects of comprehensibility and nativeness from the perspective of native-speaking English raters
ITL - International Journal of Applied Linguistics 170, 24 (2019); https://doi.org/10.1075/itl.17026.app
/content/journals/10.1075/itl.17026.app
/content/journals/10.1075/itl.17026.app
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): comprehensibility; nativeness; second language speech; vocabulary

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error