1887
  1. Home
  2. e-Journals
  3. TASK
  4. Volume 2, Issue 2
  5. Article
Volume 2, Issue 2
  • ISSN 2666-1748
  • E-ISSN: 2666-1756
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study explored the role of cognitive task complexity in designing speaking assessment tasks. A sample of 120 English learners at different proficiency levels engaged in four narrative tasks with distinct levels/types of cognitive complexity. Performances were assessed using linguistic measures and a task accomplishment rubric. Findings revealed that one of the middle-complexity tasks with moderate intrinsic, moderate germane, and low extraneous complexity elicited the best linguistic performances overall, while the lowest complexity task elicited the highest task accomplishment ratings. Focusing on distinguishing among learner abilities, however, the highest complexity task demonstrated the best discrimination. Cognitive task complexity was also found to moderate the relationship between task accomplishment ratings and linguistic measures. Implications for research into task design, learner performance, and assessment purpose are considered.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/task.21020.sas
2023-01-24
2025-07-19

  • From This Site

    /content/journals/10.1075/task.21020.sas
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Brown, J. D.
    (1980) Relative merits of four methods for scoring cloze tests. The Modern Language Journal, 641, 311–317. 10.1111/j.1540‑4781.1980.tb05198.x
     https://doi.org/10.1111/j.1540-4781.1980.tb05198.x [Google Scholar]
  2. (2005) Testing in language programs: a comprehensive guide to English language assessment. McGraw-Hill College.
     [Google Scholar]
  3. Brown, J. D., & Grüter, T.
    (2020) The same cloze for all occasions? Using the Brown (1980) cloze test for measuring proficiency in SLA research. International Review of Applied Linguistics. 10.1515/iral‑2019‑0026
     https://doi.org/10.1515/iral-2019-0026 [Google Scholar]
  4. Brown, J. D., & Hudson, T.
    (1998) The alternatives in language assessment. TESOL quarterly, 32(4), 653–675. 10.2307/3587999
     https://doi.org/10.2307/3587999 [Google Scholar]
  5. Brown, J. D., Hudson, T., Norris, J. M., & Bonk, W.
    (2002) Investigating task-based second language performance assessment. University of Hawai‘i Press.
     [Google Scholar]
  6. De Jong, N. H.
    (2018) Fluency in second language testing: Insights from different disciplines. Language Assessment Quarterly, 15(3), 237–254. 10.1080/15434303.2018.1477780
     https://doi.org/10.1080/15434303.2018.1477780 [Google Scholar]
  7. De Jong, N. H., Steinel, M., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
    (2012) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. InA. Housen, F. Kuiken & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp.121–142). John Benjamins. 10.1075/lllt.32.06jon
     https://doi.org/10.1075/lllt.32.06jon [Google Scholar]
  8. Doughty, C., & Pica, T.
    (1986) “Information gap” tasks: Do they facilitate second language acquisition?TESOL Quarterly, 20(2), 305–325. 10.2307/3586546
     https://doi.org/10.2307/3586546 [Google Scholar]
  9. Educational Testing Service
    Educational Testing Service (2019) TOEFL iBT® test speaking rubrics. https://www.ets.org/s/toefl/pdf/toefl_speaking_rubrics.pdf
     [Google Scholar]
  10. Freed, B. F., Segalowitz, N., & Dewey, D. P.
    (2004) Learning context and its effects on second language acquisition. Studies in Second Language Acquisition, 261, 275–301.
     [Google Scholar]
  11. Housen, A., De Clercq, B., Kuiken, F., & Vedder, I.
    (2019) Multiple approaches to complexity in second language research. Second language research, 35(1), 3–21. 10.1177/0267658318809765
     https://doi.org/10.1177/0267658318809765 [Google Scholar]
  12. Housen, A., Kuiken, F., & Vedder, I.
    (Eds.) (2012) Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA. John Benjamins. 10.1075/lllt.32
     https://doi.org/10.1075/lllt.32 [Google Scholar]
  13. Hunt, K. W.
    (1970) Syntactic maturity in school children and adults. Monographs of the Society for Research in Child Development, 351, 1–67. 10.2307/1165818
     https://doi.org/10.2307/1165818 [Google Scholar]
  14. Koizumi, R.
    (2012) Relationships between text length and lexical diversity measures: Can we use short texts of less than 100 tokens?Vocabulary Learning and Instruction, 1(1), 60–69. 10.7820/vli.v01.1.koizumi
     https://doi.org/10.7820/vli.v01.1.koizumi [Google Scholar]
  15. Kuiken, F., & Vedder, I.
    (2017) Functional adequacy in L2 writing: Towards a new rating scale. Language Testing, 34(3), 321–336. 10.1177/0265532216663991
     https://doi.org/10.1177/0265532216663991 [Google Scholar]
  16. Kuiken, F., Mos, M., & Vedder, I.
    (2005) Cognitive task complexity and second language writing performance. Eurosla Yearbook, 5(1), 195–222. 10.1075/eurosla.5.10kui
     https://doi.org/10.1075/eurosla.5.10kui [Google Scholar]
  17. Kuiken, F., Vedder, I., & Gilabert, R.
    (2010) Communicative adequacy and linguistic complexity in L2 writing. Communicative proficiency and linguistic development: Intersections between SLA and language testing research, EUROSLA Monograph Series11, 81–100.
     [Google Scholar]
  18. Kyle, K., Crossley, S. A., & McNamara, D. S.
    (2016) Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 33(3), 319–340. 10.1177/0265532215587391
     https://doi.org/10.1177/0265532215587391 [Google Scholar]
  19. Lee, J.
    (2020) Task closure and task complexity effects on L2 written performance. Journal of Second Language Writing, 501, 100777. 10.1016/j.jslw.2020.100777
     https://doi.org/10.1016/j.jslw.2020.100777 [Google Scholar]
  20. Long, M. H., & Norris, J. M.
    (2000) Task-based language teaching and assessment. InM. Byram (Ed.), Encyclopedia of language teaching (pp.597–603). Routledge.
     [Google Scholar]
  21. McNamara, T.
    (2000) Language testing. Oxford University Press.
     [Google Scholar]
  22. Norris, J. M.
    (2001) Identifying rating criteria for task-based EAP assessment. InT. D. Hudson & J. D. Brown (Eds.), A focus on language test development: Expanding the language proficiency construct across a variety of tests (pp.163–204). University of Hawai‘i Press.
     [Google Scholar]
  23. (2010, September). Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at theannual conference of the European Second Language Association (EUROSLA), Reggio Emilia, Italy.
     [Google Scholar]
  24. (2015) Statistical significance testing in second language research: Basic problems and some solutions. InJ. M. Norris, S. Ross & R. Schoonen (Eds.), Improving and extending quantitative reasoning in second language research (pp.95–124). Wiley-Blackwell.
     [Google Scholar]
  25. (2016) Current uses for task-based language assessment. Annual Review of Applied Linguistics, 361, 230–244. 10.1017/S0267190516000027
     https://doi.org/10.1017/S0267190516000027 [Google Scholar]
  26. Norris, J. M., & East, M.
    (2021) Task-based language assessment. InM. J. Ahmadian & M. Long (Eds.), The Cambridge Handbook of Task-Based Language Teaching. Cambridge University Press. 10.1017/9781108868327.029
     https://doi.org/10.1017/9781108868327.029 [Google Scholar]
  27. Norris, J. M., & Ortega, L.
    (2009) Towards an organic approach to investigating CAF in Instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. 10.1093/applin/amp044
     https://doi.org/10.1093/applin/amp044 [Google Scholar]
  28. Norris, J. M., Brown, J. D., Hudson, T. D., & Bonk, W.
    (2002) Examinee abilities and task difficulty in task-based L2 performance assessment. Language Testing, 19(4), 395–418. 10.1191/0265532202lt237oa
     https://doi.org/10.1191/0265532202lt237oa [Google Scholar]
  29. Pallotti, G.
    (2009) CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30(4), 590–601. 10.1093/applin/amp045
     https://doi.org/10.1093/applin/amp045 [Google Scholar]
  30. Révész, A.
    (2014) Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 351, 87–92. 10.1093/applin/amt039
     https://doi.org/10.1093/applin/amt039 [Google Scholar]
  31. Révész, A., Ekiert, M., & Torgersen, E. N.
    (2016a) The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37(6), 828–848.
     [Google Scholar]
  32. Révész, A., Michel, M., & Gilabert, R.
    (2016b) Measuring cognitive task demands using dual-task methodology, subjective self-ratings, and expert judgments: A validation study. Studies in Second Language Acquisition, 38(4), 703–737. 10.1017/S0272263115000339
     https://doi.org/10.1017/S0272263115000339 [Google Scholar]
  33. Robinson, P.
    (2001a) Task complexity, cognitive resources, and syllabus design: a triadic framework for examining task influences on SLA. InP. Robinson (Ed.), Cognition and second language instruction (pp.287–318). Cambridge University Press. 10.1017/CBO9781139524780.012
     https://doi.org/10.1017/CBO9781139524780.012 [Google Scholar]
  34. (2001b) Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 221, 27–57. 10.1093/applin/22.1.27
     https://doi.org/10.1093/applin/22.1.27 [Google Scholar]
  35. Robinson, P., & Ross, S.
    (1996) The development of task-based assessment in English for academic purposes programs. Applied Linguistics, 17(4), 455–476. 10.1093/applin/17.4.455
     https://doi.org/10.1093/applin/17.4.455 [Google Scholar]
  36. Sasayama, S.
    (2015) Validating the assumed relationship between task design, cognitive complexity, and second language task performance. Unpublished doctoral dissertation, Georgetown University.
  37. (2016) Is a ‘complex’ task really complex? Validating the assumption of cognitive task complexity. The Modern Language Journal, 100(1), 231–254. 10.1111/modl.12313
     https://doi.org/10.1111/modl.12313 [Google Scholar]
  38. Sasayama, S., & Norris, J. M.
    (2019) Unravelling cognitive task complexity: Learning from learners’ perspectives on task characteristics and L2 performance. InZ. Wen & M. J. Ahmadian (Eds.), Researching L2 task performance and pedagogy: In honour of Peter Skehan (pp.95–132). John Benjamins. 10.1075/tblt.13.06sas
     https://doi.org/10.1075/tblt.13.06sas [Google Scholar]
  39. Sasayama, S., Garcia Gomez, P., & Norris, J. M.
    (2021) Designing efficient L2 writing assessment tasks for low-proficiency learners of English. TOEFL Research Report. Princeton, NJ: Educational Testing Service. 10.1002/ets2.12341
     https://doi.org/10.1002/ets2.12341 [Google Scholar]
  40. Skehan, P.
    (1998) A cognitive approach to language learning. Oxford University Press.
     [Google Scholar]
  41. (2009) Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 301, 510–532. 10.1093/applin/amp047
     https://doi.org/10.1093/applin/amp047 [Google Scholar]
  42. Steiger, J. H.
    (1980) Tests for comparing elements of a correlation matrix. Psychological Bulletin, 871, 245–251. 10.1037/0033‑2909.87.2.245
     https://doi.org/10.1037/0033-2909.87.2.245 [Google Scholar]
  43. Sweller, J.
    (1994) Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 41, 295–312. 10.1016/0959‑4752(94)90003‑5
     https://doi.org/10.1016/0959-4752(94)90003-5 [Google Scholar]
  44. Treffers-Daller, J., Parslow, P., & Williams, S.
    (2018) Back to basics: How measures of lexical diversity can help discriminate between CEFR levels. Applied Linguistics, 39(3), 302–327.
     [Google Scholar]
  45. Van Gorp, K., & Deygers, B.
    (2014) Task-based language assessment. InA. Kunnan (Ed.), The companion to language assessment, 21 (pp.578–593). John Wiley & Sons.
     [Google Scholar]
  46. Vasylets, O., Gilabert, R., & Manchon, R. M.
    (2020) Task modality, communicative adequacy and CAF measures: The moderating role of task complexity. InManchon, R. M. (Ed.), Writing and language learning: Advancing research agendas (pp.183–206). John Benjamins. 10.1075/lllt.56.08vas
     https://doi.org/10.1075/lllt.56.08vas [Google Scholar]
  47. Xi, X., & Norris, J. M.
    (Eds.) (2021) Assessing academic English for higher education admissions. Routledge. 10.4324/9781351142403
     https://doi.org/10.4324/9781351142403 [Google Scholar]
  48. Yu, G.
    (2010) Lexical diversity in writing and speaking task performances. Applied Linguistics, 31(2), 236–259. 10.1093/applin/amp024
     https://doi.org/10.1093/applin/amp024 [Google Scholar]
/content/journals/10.1075/task.21020.sas
Loading
Designing speaking tasks for different assessment goals
task 2, 184 (2022); https://doi.org/10.1075/task.21020.sas
/content/journals/10.1075/task.21020.sas
/content/journals/10.1075/task.21020.sas
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): assessment task design; CALF measures; cognitive task complexity; discrimination; task accomplishment

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error