1887
image of Assessing Chinese EFL learners’ speaking proficiency levels
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study elicits 120 recordings as research data from the Test for English Majors Band 4 (TEM-4) Oral Test. We investigate the distinguishing cohesive devices across and between four pairs of speaking proficiency levels and the predictive cohesive devices for L2 speaking proficiency by examining all cohesion indices in TAACO for an independent speaking task. Results show that 15 local, 3 global, and 6 text cohesion indices distinguish across speaking proficiency levels. Besides, cohesion indices vary in differentiating powers at certain levels. In addition, 7 local, 5 global, and 1 text cohesion indices significantly correlate to L2 speaking proficiency levels. The regression model containing 2 local and 3 text cohesion indices explains 63.8% of the variance in predicting L2 speaking proficiency levels. These findings hold some implications for L2 speaking pedagogy and test assessment.

Loading

Article metrics loading...

/content/journals/10.1075/jsls.00044.xu
2025-03-11
2025-03-22
Loading full text...

Full text loading...

References

  1. Abdi Tabari, M., & Johnson, M. D.
    (2023) Exploring new insights into the role of cohesive devices in written academic genres. Assessing Writing, , . 10.1016/j.asw.2023.100749
    https://doi.org/10.1016/j.asw.2023.100749 [Google Scholar]
  2. Abdi Tabari, M., & Wind, A. M.
    (2023) Dynamic development of cohesive devices in English as a second language writing. International Review of Applied Linguistics in Language Teaching, (). 10.1515/iral‑2022‑0205
    https://doi.org/10.1515/iral-2022-0205 [Google Scholar]
  3. Chiang, S. Y.
    (1999) Assessing grammatical and textual features in L2 writing samples: The case of French as a foreign language. The Modern Language Journal, (), –. 10.1111/0026‑7902.00017
    https://doi.org/10.1111/0026-7902.00017 [Google Scholar]
  4. Covington, M. A., & McFall, J. D.
    (2010) Cutting the gordian knot: The moving-average type–token ratio (MATTR). Journal of Quantitative Linguistics, (), –. 10.1080/09296171003643098
    https://doi.org/10.1080/09296171003643098 [Google Scholar]
  5. Crossley, S. A., & Kim, Y.
    (2019) Text integration and speaking proficiency: Linguistic, individual differences, and strategy use considerations. Language Assessment Quarterly, (), –. 10.1080/15434303.2019.1628239
    https://doi.org/10.1080/15434303.2019.1628239 [Google Scholar]
  6. Crossley, S. A., & McNamara, D. S.
    (2010) Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, (), –. 10.1111/j.1467‑9817.2010.01449.x
    https://doi.org/10.1111/j.1467-9817.2010.01449.x [Google Scholar]
  7. Crossley, S. A., Kyle, K., & Dascalu, M.
    (2019) The tool for the automatic analysis of cohesion 2.0: Integrating semantic similarity and text overlap. Behavior Research Methods, (), –. 10.3758/s13428‑018‑1142‑4
    https://doi.org/10.3758/s13428-018-1142-4 [Google Scholar]
  8. Crossley, S. A., Kyle, K., & McNamara, D. S.
    (2016a) The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing, , –. 10.1016/j.jslw.2016.01.003
    https://doi.org/10.1016/j.jslw.2016.01.003 [Google Scholar]
  9. (2016b) The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods, (), –. 10.3758/s13428‑015‑0651‑7
    https://doi.org/10.3758/s13428-015-0651-7 [Google Scholar]
  10. Crossley, S. A., & McNamara, D. S.
    (2013) Applications of text analysis tools for spoken response grading. Language Learning & Technology, (), –. 10125/44329
    https://doi.org/10125/44329 [Google Scholar]
  11. Crossley, S. A., Clevinger, A., & Kim, Y.
    (2014) The role of lexical properties and cohesive devices in text integration and their effect on human ratings of speaking proficiency. Language Assessment Quarterly, (), –. 10.1080/15434303.2014.926905
    https://doi.org/10.1080/15434303.2014.926905 [Google Scholar]
  12. Foster, P., Tonkyn, A., & Wigglesworth, G.
    (2000) Measuring spoken language: A unit for all reasons. Applied Linguistics, , –. 10.1093/applin/21.3.354
    https://doi.org/10.1093/applin/21.3.354 [Google Scholar]
  13. Gao, M., & Liu, Y.
    (2023) An analysis of the distinguishing features of discourse performance in an online diagnostic speaking test. Journal of China Examinations, (), –.
    [Google Scholar]
  14. Graesser, A. C., McNamara, D. S., & Louwerse, M. M.
    (2003) What do readers need to learn in order to process coherence relations in narrative and expository text. InA. P. Sweet & C. E. Snow (Eds.), Rethinking Reading Comprehension (pp.–). New York, NY: Guilford Press.
    [Google Scholar]
  15. Gu, L., & Hsieh, C.-N.
    (2019) Distinguishing features of young english language learners’ oral performance. Language Assessment Quarterly, (), –. 10.1080/15434303.2019.1605518
    https://doi.org/10.1080/15434303.2019.1605518 [Google Scholar]
  16. Guo, L., Crossley, S. A., & McNamara, D. S.
    (2013) Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, (), –. 10.1016/j.asw.2013.05.002
    https://doi.org/10.1016/j.asw.2013.05.002 [Google Scholar]
  17. Halliday, M. A. K., & Hasan, R.
    (1976) Cohesion in English. London: Longman.
    [Google Scholar]
  18. (1985) Language, context and text: Aspects of language in social-semiotic perspective. Deakin University Press.
    [Google Scholar]
  19. Iwashita, N., May, L., & Moore, P.
    (2017) Features of discourse and lexical richness at different performance levels in the Aptis speaking test. ARAGs Research Reports, AR-G/2017/002. British Council. https://www.britishcouncil.org/sites/default/files/iwashita_et_al_layout_1_revised.pdf
  20. Jin, Y., & Fan, J.
    (2011) Test for English majors (TEM) in China. Language Testing, (), –. 10.1177/0265532211414852
    https://doi.org/10.1177/0265532211414852 [Google Scholar]
  21. Lee, C., Ge, H., & Chung, E.
    (2021) What linguistic features distinguish and predict L2 writing quality? A study of examination scripts written by adolescent Chinese learners of English in Hong Kong. System, , . 10.1016/j.system.2021.102461
    https://doi.org/10.1016/j.system.2021.102461 [Google Scholar]
  22. Lin, Y.-L., Tsai, Y.-S., & Hsieh, C.-Y.
    (2023) Discourse competence across band scores: An analysis of speaking performance in the general english proficiency test. International Review of Applied Linguistics in Language Teaching, (). 10.1515/iral‑2022‑0154
    https://doi.org/10.1515/iral-2022-0154 [Google Scholar]
  23. Liu, M., & Braine, G.
    (2005) Cohesive features in argumentative writing produced by Chinese undergraduates. System, (), –. 10.1016/j.system.2005.02.002
    https://doi.org/10.1016/j.system.2005.02.002 [Google Scholar]
  24. Liu, Q., Zhou, S.
    (2024) Exploring Interaction Patterns in Web-Based Collaborative Writing Tasks Among Advanced L2 Learners: A Case Study Using Feishu Platform. In: Kubincová, Z., Emerging Technologies for Education. SETE 2023. Lecture Notes in Computer Science, vol 14607. Springer, Singapore. 10.1007/978‑981‑97‑4246‑2_4
    https://doi.org/10.1007/978-981-97-4246-2_4 [Google Scholar]
  25. Lu, X.
    (2012) The relationship of lexical richness to the quality of ESL learners’ oral narratives. The Modern Language Journal, (), –. 10.1111/j.1540‑4781.2011.01232_1.x
    https://doi.org/10.1111/j.1540-4781.2011.01232_1.x [Google Scholar]
  26. McCarthy, P. M., & Jarvis, S.
    (2010) MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, (), –. 10.3758/BRM.42.2.381
    https://doi.org/10.3758/BRM.42.2.381 [Google Scholar]
  27. McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C.
    (2010) Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, (), –. 10.1080/01638530902959943
    https://doi.org/10.1080/01638530902959943 [Google Scholar]
  28. Tyler, A.
    (1992) Discourse structure and the perception of incoherence in international teaching assistants’ spoken discourse. TESOL Quarterly, (), . 10.2307/3586870
    https://doi.org/10.2307/3586870 [Google Scholar]
  29. Tywoniw, R., & Crossley, S.
    (2019) The effect of cohesive features in integrated and independent L2 writing quality and text classification. Language Education and Assessment, (), –. 10.29140/lea.v2n3.151
    https://doi.org/10.29140/lea.v2n3.151 [Google Scholar]
  30. Vögelin, C., Jansen, T., Keller, S. D., Machts, N., & Möller, J.
    (2019) The influence of lexical features on teacher judgements of ESL argumentative essays. Assessing Writing, , –. 10.1016/j.asw.2018.12.003
    https://doi.org/10.1016/j.asw.2018.12.003 [Google Scholar]
  31. Wang, T., & Zhou, D.
    (2023) Analyzing the distinguishing language features of Chinese EFL learners’ oral expression. Modern Foreign Languages, (), –.
    [Google Scholar]
  32. Witte, S. P., & Faigley, L.
    (1981) Coherence, cohesion, and writing quality. College Composition and Communication, (), –. 10.58680/ccc198115912
    https://doi.org/10.58680/ccc198115912 [Google Scholar]
  33. Witten, I. H., Frank, E., & Hall, M. A.
    (2011) Data mining: Practical machine learning tools and techniques. Burlington, MA: Morgan Kaufmann.
    [Google Scholar]
  34. Yang, W., & Sun, Y.
    (2012) The use of cohesive devices in argumentative writing by Chinese EFL learners at different proficiency levels. Linguistics and Education, (): –. 10.1016/j.linged.2011.09.004
    https://doi.org/10.1016/j.linged.2011.09.004 [Google Scholar]
  35. Yu, Hongliang
    (2005) An exploratory study of narrative structure in Chinese tertiary EFL learners’ monologic production. Foreign Language Teaching and Research Press.
    [Google Scholar]
/content/journals/10.1075/jsls.00044.xu
Loading
/content/journals/10.1075/jsls.00044.xu
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error