1887
Volume 2, Issue 2
  • ISSN 2666-1748
  • E-ISSN: 2666-1756
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study explored the role of cognitive task complexity in designing speaking assessment tasks. A sample of 120 English learners at different proficiency levels engaged in four narrative tasks with distinct levels/types of cognitive complexity. Performances were assessed using linguistic measures and a task accomplishment rubric. Findings revealed that one of the middle-complexity tasks with moderate intrinsic, moderate germane, and low extraneous complexity elicited the best linguistic performances overall, while the lowest complexity task elicited the highest task accomplishment ratings. Focusing on distinguishing among learner abilities, however, the highest complexity task demonstrated the best discrimination. Cognitive task complexity was also found to moderate the relationship between task accomplishment ratings and linguistic measures. Implications for research into task design, learner performance, and assessment purpose are considered.

Loading

Article metrics loading...

/content/journals/10.1075/task.21020.sas
2023-01-24
2023-02-03
Loading full text...

Full text loading...

References

  1. Brown, J. D.
    (1980) Relative merits of four methods for scoring cloze tests. The Modern Language Journal, 641, 311–317. 10.1111/j.1540‑4781.1980.tb05198.x
    https://doi.org/10.1111/j.1540-4781.1980.tb05198.x [Google Scholar]
  2. (2005) Testing in language programs: a comprehensive guide to English language assessment. McGraw-Hill College.
    [Google Scholar]
  3. Brown, J. D., & Grüter, T.
    (2020) The same cloze for all occasions? Using the Brown (1980) cloze test for measuring proficiency in SLA research. International Review of Applied Linguistics. 10.1515/iral‑2019‑0026
    https://doi.org/10.1515/iral-2019-0026 [Google Scholar]
  4. Brown, J. D., & Hudson, T.
    (1998) The alternatives in language assessment. TESOL quarterly, 32(4), 653–675. 10.2307/3587999
    https://doi.org/10.2307/3587999 [Google Scholar]
  5. Brown, J. D., Hudson, T., Norris, J. M., & Bonk, W.
    (2002) Investigating task-based second language performance assessment. University of Hawai‘i Press.
    [Google Scholar]
  6. De Jong, N. H.
    (2018) Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly, 15(3), 237–254. 10.1080/15434303.2018.1477780
    https://doi.org/10.1080/15434303.2018.1477780 [Google Scholar]
  7. De Jong, N. H., Steinel, M., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
    (2012) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. InA. Housen, F. Kuiken & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp.121–142). John Benjamins. 10.1075/lllt.32.06jon
    https://doi.org/10.1075/lllt.32.06jon [Google Scholar]
  8. Doughty, C., & Pica, T.
    (1986) “Information gap” tasks: Do they facilitate second language acquisition?TESOL Quarterly, 20(2), 305–325. 10.2307/3586546
    https://doi.org/10.2307/3586546 [Google Scholar]
  9. Educational Testing Service
    Educational Testing Service (2019) TOEFL iBT® test speaking rubrics. https://www.ets.org/s/toefl/pdf/toefl_speaking_rubrics.pdf
    [Google Scholar]
  10. Freed, B. F., Segalowitz, N., & Dewey, D. P.
    (2004) Learning context and its effects on second language acquisition. Studies in Second Language Acquisition, 261, 275–301.
    [Google Scholar]
  11. Housen, A., De Clercq, B., Kuiken, F., & Vedder, I.
    (2019) Multiple approaches to complexity in second language research. Second language research, 35(1), 3–21. 10.1177/0267658318809765
    https://doi.org/10.1177/0267658318809765 [Google Scholar]
  12. Housen, A., Kuiken, F., & Vedder, I.
    (Eds.) (2012) Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA. John Benjamins. 10.1075/lllt.32
    https://doi.org/10.1075/lllt.32 [Google Scholar]
  13. Hunt, K. W.
    (1970) Syntactic maturity in school children and adults. Monographs of the Society for Research in Child Development, 351, 1–67. 10.2307/1165818
    https://doi.org/10.2307/1165818 [Google Scholar]
  14. Koizumi, R.
    (2012) Relationships between text length and lexical diversity measures: Can we use short texts of less than 100 tokens?Vocabulary Learning and Instruction, 1(1), 60–69. 10.7820/vli.v01.1.koizumi
    https://doi.org/10.7820/vli.v01.1.koizumi [Google Scholar]
  15. Kuiken, F., & Vedder, I.
    (2017) Functional adequacy in L2 writing: Towards a new rating scale. Language Testing, 34(3), 321–336. 10.1177/0265532216663991
    https://doi.org/10.1177/0265532216663991 [Google Scholar]
  16. Kuiken, F., Mos, M., & Vedder, I.
    (2005) Cognitive task complexity and second language writing performance. Eurosla Yearbook, 5(1), 195–222. 10.1075/eurosla.5.10kui
    https://doi.org/10.1075/eurosla.5.10kui [Google Scholar]
  17. Kuiken, F., Vedder, I., & Gilabert, R.
    (2010) Communicative adequacy and linguistic complexity in L2 writing. Communicative proficiency and linguistic development: Intersections between SLA and language testing research, EUROSLA Monograph Series11, 81–100.
    [Google Scholar]
  18. Kyle, K., Crossley, S. A., & McNamara, D. S.
    (2016) Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 33(3), 319–340. 10.1177/0265532215587391
    https://doi.org/10.1177/0265532215587391 [Google Scholar]
  19. Lee, J.
    (2020) Task closure and task complexity effects on L2 written performance. Journal of Second Language Writing, 501, 100777. 10.1016/j.jslw.2020.100777
    https://doi.org/10.1016/j.jslw.2020.100777 [Google Scholar]
  20. Long, M. H., & Norris, J. M.
    (2000) Task-based language teaching and assessment. InM. Byram (Ed.), Encyclopedia of language teaching (pp.597–603). Routledge.
    [Google Scholar]
  21. McNamara, T.
    (2000) Language testing. Oxford University Press.
    [Google Scholar]
  22. Norris, J. M.
    (2001) Identifying rating criteria for task-based EAP assessment. InT. D. Hudson & J. D. Brown (Eds.), A focus on language test development: Expanding the language proficiency construct across a variety of tests (pp.163–204). University of Hawai‘i Press.
    [Google Scholar]
  23. (2010, September). Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at theannual conference of the European Second Language Association (EUROSLA), Reggio Emilia, Italy.
    [Google Scholar]
  24. (2015) Statistical significance testing in second language research: Basic problems and some solutions. InJ. M. Norris, S. Ross & R. Schoonen (Eds.), Improving and extending quantitative reasoning in second language research (pp.95–124). Wiley-Blackwell.
    [Google Scholar]
  25. (2016) Current uses for task-based language assessment. Annual Review of Applied Linguistics, 361, 230–244. 10.1017/S0267190516000027
    https://doi.org/10.1017/S0267190516000027 [Google Scholar]
  26. Norris, J. M., & East, M.
    (2021) Task-based language assessment. InM. J. Ahmadian & M. Long (Eds.), The Cambridge Handbook of Task-Based Language Teaching. Cambridge University Press. 10.1017/9781108868327.029
    https://doi.org/10.1017/9781108868327.029 [Google Scholar]
  27. Norris, J. M., & Ortega, L.
    (2009) Towards an organic approach to investigating CAF in Instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. 10.1093/applin/amp044
    https://doi.org/10.1093/applin/amp044 [Google Scholar]
  28. Norris, J. M., Brown, J. D., Hudson, T. D., & Bonk, W.
    (2002) Examinee abilities and task difficulty in task-based L2 performance assessment. Language Testing, 19(4), 395–418. 10.1191/0265532202lt237oa
    https://doi.org/10.1191/0265532202lt237oa [Google Scholar]
  29. Pallotti, G.
    (2009) CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30(4), 590–601. 10.1093/applin/amp045
    https://doi.org/10.1093/applin/amp045 [Google Scholar]
  30. Révész, A.
    (2014) Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 351, 87–92. 10.1093/applin/amt039
    https://doi.org/10.1093/applin/amt039 [Google Scholar]
  31. Révész, A., Ekiert, M., & Torgersen, E. N.
    (2016a) The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37(6), 828–848.
    [Google Scholar]
  32. Révész, A., Michel, M., & Gilabert, R.
    (2016b) Measuring cognitive task demands using dual-task methodology, subjective self-ratings, and expert judgments: A validation study. Studies in Second Language Acquisition, 38(4), 703–737. 10.1017/S0272263115000339
    https://doi.org/10.1017/S0272263115000339 [Google Scholar]
  33. Robinson, P.
    (2001a) Task complexity, cognitive resources, and syllabus design: a triadic framework for examining task influences on SLA. InP. Robinson (Ed.), Cognition and second language instruction (pp.287–318). Cambridge University Press. 10.1017/CBO9781139524780.012
    https://doi.org/10.1017/CBO9781139524780.012 [Google Scholar]
  34. (2001b) Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 221, 27–57. 10.1093/applin/22.1.27
    https://doi.org/10.1093/applin/22.1.27 [Google Scholar]
  35. Robinson, P., & Ross, S.
    (1996) The development of task-based assessment in English for academic purposes programs. Applied Linguistics, 17(4), 455–476. 10.1093/applin/17.4.455
    https://doi.org/10.1093/applin/17.4.455 [Google Scholar]
  36. Sasayama, S.
    (2015) Validating the assumed relationship between task design, cognitive complexity, and second language task performance. Unpublished doctoral dissertation, Georgetown University.
  37. (2016) Is a ‘complex’ task really complex? Validating the assumption of cognitive task complexity. The Modern Language Journal, 100(1), 231–254. 10.1111/modl.12313
    https://doi.org/10.1111/modl.12313 [Google Scholar]
  38. Sasayama, S., & Norris, J. M.
    (2019) Unravelling cognitive task complexity: Learning from learners’ perspectives on task characteristics and L2 performance. InZ. Wen & M. J. Ahmadian (Eds.), Researching L2 task performance and pedagogy: In honour of Peter Skehan (pp.95–132). John Benjamins. 10.1075/tblt.13.06sas
    https://doi.org/10.1075/tblt.13.06sas [Google Scholar]
  39. Sasayama, S., Garcia Gomez, P., & Norris, J. M.
    (2021) Designing efficient L2 writing assessment tasks for low-proficiency learners of English. TOEFL Research Report. Princeton, NJ: Educational Testing Service. 10.1002/ets2.12341
    https://doi.org/10.1002/ets2.12341 [Google Scholar]
  40. Skehan, P.
    (1998) A cognitive approach to language learning. Oxford University Press.
    [Google Scholar]
  41. (2009) Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 301, 510–532. 10.1093/applin/amp047
    https://doi.org/10.1093/applin/amp047 [Google Scholar]
  42. Steiger, J. H.
    (1980) Tests for comparing elements of a correlation matrix. Psychological Bulletin, 871, 245–251. 10.1037/0033‑2909.87.2.245
    https://doi.org/10.1037/0033-2909.87.2.245 [Google Scholar]
  43. Sweller, J.
    (1994) Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 41, 295–312. 10.1016/0959‑4752(94)90003‑5
    https://doi.org/10.1016/0959-4752(94)90003-5 [Google Scholar]
  44. Treffers-Daller, J., Parslow, P., & Williams, S.
    (2018) Back to basics: How measures of lexical diversity can help discriminate between CEFR levels. Applied Linguistics, 39(3), 302–327.
    [Google Scholar]
  45. Van Gorp, K., & Deygers, B.
    (2014) Task-based language assessment. InA. Kunnan (Ed.), The companion to language assessment, 21 (pp.578–593). John Wiley & Sons.
    [Google Scholar]
  46. Vasylets, O., Gilabert, R., & Manchon, R. M.
    (2020) Task modality, communicative adequacy and CAF measures: The moderating role of task complexity. InManchon, R. M. (Ed.), Writing and language learning: Advancing research agendas (pp.183–206). John Benjamins. 10.1075/lllt.56.08vas
    https://doi.org/10.1075/lllt.56.08vas [Google Scholar]
  47. Xi, X., & Norris, J. M.
    (Eds.) (2021) Assessing academic English for higher education admissions. Routledge. 10.4324/9781351142403
    https://doi.org/10.4324/9781351142403 [Google Scholar]
  48. Yu, G.
    (2010) Lexical diversity in writing and speaking task performances. Applied Linguistics, 31(2), 236–259. 10.1093/applin/amp024
    https://doi.org/10.1093/applin/amp024 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/task.21020.sas
Loading
/content/journals/10.1075/task.21020.sas
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error