1887
  1. Home
  2. e-Journals
  3. TASK
  4. Volume 2, Issue 1
  5. Article
Volume 2, Issue 1
  • ISSN 2666-1748
  • E-ISSN: 2666-1756
EPUB
PDF

Abstract

Abstract

Linguistic performance elicited by language tasks has generally been operationalized in terms of complexity, accuracy, and fluency (CAF). However, this study argues that assessment of L2 proficiency is impossible without taking into account the adequacy and efficacy of L2 performance. To that end, we developed a rating scale for measuring functional adequacy (FA). In order to investigate the validity, reliability, and applicability of the rating scale, a number of studies are reviewed in which FA was assessed by both expert and non-expert raters, in different learning contexts, for L2 and L1, involving various source and target languages, proficiency levels, task types and modalities. We discuss perspectives and challenges for the use of the FA rating scale, particularly with regard to task-based language assessment (TBLA).

© John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/task.00013.kui
2022-06-20
2024-09-17

  • From This Site

    /content/journals/10.1075/task.00013.kui
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/task.00013.kui.html?itemId=/content/journals/10.1075/task.00013.kui&mimeType=html&fmt=ahah

References

  1. Bachman, L. F.
    (2002) Some reflections on task-based language performance assessment. Language Testing, 19(4), 453–476. 10.1191/0265532202lt240oa
     https://doi.org/10.1191/0265532202lt240oa [Google Scholar]
  2. Becker, A.
    (2018) Not to scale? An argument-based inquiry into the validity of an L2 writing scale. Assessing Writing, 37, 1–12. 10.1016/j.asw.2018.01.001
     https://doi.org/10.1016/j.asw.2018.01.001 [Google Scholar]
  3. Bridgeman, B., Powers, D., Stone, E., & Mollaun, P.
    (2012) TOEFL iBT speaking test scores as indicators of oral communicative language proficiency. Language Testing29(1), 91–108. 10.1177/0265532211411078
     https://doi.org/10.1177/0265532211411078 [Google Scholar]
  4. Bulté, B., & Housen, A.
    (2012) Defining and operationalising L2 complexity. InA. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA (pp. 21–46). John Benjamins. 10.1075/lllt.32.02bul
     https://doi.org/10.1075/lllt.32.02bul [Google Scholar]
  5. Council of Europe
    Council of Europe (2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
     [Google Scholar]
  6. De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
    (2012a) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. InA. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA (pp. 121–142). John Benjamins. 10.1075/lllt.32.06jon
     https://doi.org/10.1075/lllt.32.06jon [Google Scholar]
  7. (2012b) Facets of speaking proficiency. Studies in Second Language Acquisition, 34(1), 5–34. 10.1017/S0272263111000489
     https://doi.org/10.1017/S0272263111000489 [Google Scholar]
  8. Del Bono, F.
    (2019) Aspetti pragmatici nella valutazione di testi scritti: Uno studio sull’adeguatezza funzionale in italiano L2. In: E. Nuzzo, & I. Vedder (Eds.), Lingua in contesto. La prospettiva pragmatica. Studi AItLA9 (pp. 231–244). Associazione Italiana di Linguistica Applicata (AitLA).
     [Google Scholar]
  9. (2020) L’utilizzo delle scale dell’adeguatezza funzionale su testi narrativi in L2: Uno studio esplorativo sugli effetti del task design. In: E. Nuzzo, E. Santoro, & I. Vedder (Eds.), Valutazione e misurazione delle produzioni orali e scritte in italiano lingua seconda (pp. 71–82). Franco Cesati Editore.
     [Google Scholar]
  10. De Meo, A., Maffia, M., & Vitale, G.
    (2019) La competenze scritta in italiano L2 di apprendenti vulnerabili. Due scale di valutazione a confronto. EL.LE, 8(3), 637–654. 10.30687/ELLE/2280‑6792/2019/03/007
     https://doi.org/10.30687/ELLE/2280-6792/2019/03/007 [Google Scholar]
  11. Ekiert, M., Lampropoulou, S., Révész, A., & Torgersen, E.
    (2018) The effects of task type and L2 proficiency on discourse appropriacy in oral task performance. InN. Taguchi, & Y-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 247–264). John Benjamins. 10.1075/tblt.10.10eki
     https://doi.org/10.1075/tblt.10.10eki [Google Scholar]
  12. Faone, S., & Pagliara, F.
    (2017) How to assess L2 information-gap tasks through FA rating scales. Paper presented atTBLT2017.
     [Google Scholar]
  13. González-Lloret, M.
    (2016) A practical guide to integrating technology into task-based language teaching. Georgetown University Press. 10.6035/LanguageV.2017.9.9
     https://doi.org/10.6035/LanguageV.2017.9.9 [Google Scholar]
  14. Grice, H. P.
    (1975) Logic and conversation. InP. Cole, & J. L. Morgan (Eds.), Speech acts (pp. 41–58). Academic Press.
     [Google Scholar]
  15. Gurzynski-Weiss, L., & IATBLT
    (n.d.). The TBLT Language Learning Task Bank. https://tblt.indiana.edu
     [Google Scholar]
  16. Herraiz Martínez, A.
    (2018) Functional adequacy: The influence of English-medium instruction, English proficiency, and previous language learning experiences. Doctoral dissertation, Universitat Jaume I, Castellón de la Plana.
  17. Herraiz Martínez, A., & Alcón Soler, E.
    (2019) Pragmatic outcomes in the English-medium instruction context. Applied Pragmatics, 1(1), 68–91. 10.1075/ap.00004.her
     https://doi.org/10.1075/ap.00004.her [Google Scholar]
  18. Housen, A., Kuiken, F., & Vedder, I.
    (Eds.) (2012) Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA. John Benjamins. 10.1075/lllt.32
     https://doi.org/10.1075/lllt.32 [Google Scholar]
  19. Knoch, U.
    (2007) ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence. Assessing Writing, 12(2), 108-128. 10.1016/j.asw.2007.07.002
     https://doi.org/10.1016/j.asw.2007.07.002 [Google Scholar]
  20. (2009) Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275–304. 10.1177/0265532208101008
     https://doi.org/10.1177/0265532208101008 [Google Scholar]
  21. (2011) Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from?Assessing Writing, 16(2), 81–96. 10.1016/j.asw.2011.02.003
     https://doi.org/10.1016/j.asw.2011.02.003 [Google Scholar]
  22. Kuiken, F., & Vedder, I.
    (2012) Speaking and writing tasks and their effects on second language performance. InS. M. Gass, & A. Mackey (Eds.), The Routledge handbook of second language acquisition (pp. 364–377). Routledge. 10.4324/9780203808184
     https://doi.org/10.4324/9780203808184 [Google Scholar]
  23. (2014) Rating written performance: What do raters do and why?Language Testing, 31(3), 329–348. 10.1177/0265532214526174
     https://doi.org/10.1177/0265532214526174 [Google Scholar]
  24. (2017) Functional adequacy in L2 writing. Towards a new rating scale. Language Testing, 34(3), 321–336. 10.1177/0265532216663991
     https://doi.org/10.1177/0265532216663991 [Google Scholar]
  25. (2018) Assessing functional adequacy of L2 performance in a task-based approach. InN. Taguchi, & Y-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 265–286). John Benjamins. 10.1075/tblt.10.11kui
     https://doi.org/10.1075/tblt.10.11kui [Google Scholar]
  26. (2021) Scoring approaches: Scales/rubrics. InP. Winke, & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 125–134). Routledge.
     [Google Scholar]
  27. Kuiken, F., Vedder, I., & Gilabert, R.
    (2010) Communicative adequacy and linguistic complexity in L2 writing. InI. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). European Second Language Association.
     [Google Scholar]
  28. Long, M. H.
    (2015) Second language acquisition and task-based language teaching. Wiley Blackwell.
     [Google Scholar]
  29. (2016) In defense of tasks and TBLT: Nonissues and real issues. Annual Review of Applied Linguistics, 36, 5–33. 10.1017/S0267190515000057
     https://doi.org/10.1017/S0267190515000057 [Google Scholar]
  30. Luoma, S.
    (2004) Assessing speaking. Cambridge University Press. 10.1017/CBO9780511733017
     https://doi.org/10.1017/CBO9780511733017 [Google Scholar]
  31. Martín Laguna, S.
    (forthcoming). Testing functional adequacy in L2 writing across languages, levels and tasks. Universitat Jaume I, Castellón de la Plana.
     [Google Scholar]
  32. McNamara, T., & Roever, C.
    (2007) Testing: The social dimension. Blackwell. 10.1111/j.1473‑4192.2006.00117.x
     https://doi.org/10.1111/j.1473-4192.2006.00117.x [Google Scholar]
  33. Norris, J. M.
    (2016) Current uses for task-based language assessment. Annual Review of Applied Linguistics, 36, 23–244. 10.1017/S0267190516000027
     https://doi.org/10.1017/S0267190516000027 [Google Scholar]
  34. Norris, J. M., & Ortega, L.
    (2003) Defining and measuring SLA. InC. J. Doughty, & M. H. Long (Eds.), The handbook of seond language acquisition (pp. 717–761). Blackwell. 10.1002/9780470756492.ch21
     https://doi.org/10.1002/9780470756492.ch21 [Google Scholar]
  35. (2009) Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. 10.1093/applin/amp044
     https://doi.org/10.1093/applin/amp044 [Google Scholar]
  36. Nuzzo, E., & Bove, G.
    (2020) Assessing functional adequacy across tasks: A comparison of learners’ and speakers’ written texts. E-JournALL, 7(2), 9–27. 10.21283/2376905X.12.175
     https://doi.org/10.21283/2376905X.12.175 [Google Scholar]
  37. Orrù, P.
    (2019) Misurare l’adeguatezza funzionale in testi scritti di apprendenti di italiano L2. Italiano LinguaDue, 1, 45–58. 10.13130/2037‑3597/11843
     https://doi.org/10.13130/2037-3597/11843 [Google Scholar]
  38. Orrù, P., & Foti, E.
    (2020) Coerenza e coesione nella valutazioni dell’adeguatezza funzionale: Un confronto tra i giudizi dei valuatori. In: E. Nuzzo, E. Santoro, & I. Vedder (Eds.), Valutazione e misurazione delle produzioni orali e scritte in italiano lingua seconda (pp. 83–92). Franco Cesati Editore.
     [Google Scholar]
  39. Ortega, L.
    (2003) Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college level L2 writing. Applied Linguistics, 24(4), 492–518. 10.1093/applin/24.4.492
     https://doi.org/10.1093/applin/24.4.492 [Google Scholar]
  40. Pallotti, G.
    (2009) CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30(4), 590–601. 10.1093/applin/amp045
     https://doi.org/10.1093/applin/amp045 [Google Scholar]
  41. (2017a) Applying the interlanguage approach to language teaching. IRAL, 55(4), 393–412. 10.1515/iral‑2017‑0145
     https://doi.org/10.1515/iral-2017-0145 [Google Scholar]
  42. (2017b) Osservare l’interlingua. Percorsi di educazione linguistica efficace per ridurre le diseguaglianze. InM. Vedonelli (Ed.), L’italiano dei nuovi italiani. Atti del XIX Convegno Nazionale GISCEL (pp. 505–520). Aracne.
     [Google Scholar]
  43. (2017c) Une application des recherches sur l’interlangue aux contextes d’enseignement. Le Français dans le monde, 61, 109–120.
     [Google Scholar]
  44. (2019) Assessing tasks: The case of interactional difficulty. Applied Linguistics, 40(1), 176–197. 10.1093/applin/amx020
     https://doi.org/10.1093/applin/amx020 [Google Scholar]
  45. Pallotti, G., & Brezina, V.
    (2019) Morphological complexity in written L2 texts. Second Language Research, 35(1), 99–119. 10.1177/0267658316643125
     https://doi.org/10.1177/0267658316643125 [Google Scholar]
  46. Paquot, M.
    (2018) Phraseological competence: A missing component in university entrance language tests? Insights from a study of EFL learners’ use of statistical collocations. Language Assessment Quarterly, 15(1), 29–43. 10.1080/15434303.2017.1405421
     https://doi.org/10.1080/15434303.2017.1405421 [Google Scholar]
  47. (2019) The phraseological dimension in interlanguage complexity research. Second Language Research, 35(1), 121–145. 10.1177/0267658317694221
     https://doi.org/10.1177/0267658317694221 [Google Scholar]
  48. Phakiti, A.
    (2020) Likert-type scale construction. InP. Winke, & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 102–114). Routledge. 10.4324/9781351034784‑12
     https://doi.org/10.4324/9781351034784-12 [Google Scholar]
  49. Pill, J., & Smart, C.
    (2020) Rating: Behavior and training. InP. Winke, & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 135–144). Routledge. 10.4324/9781351034784‑15
     https://doi.org/10.4324/9781351034784-15 [Google Scholar]
  50. Révész, A., Ekiert, M., & Torgersen, E.
    (2016) The effects of complexity, accuracy and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37(6), 828–848. 10.1093/applin/amu069
     https://doi.org/10.1093/applin/amu069 [Google Scholar]
  51. Révész, A., & Brunfaut, T.
    (2021) Validating assessments for research purposes. InP. Winke, & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 21–32). Routledge.
     [Google Scholar]
  52. Rezaei, A. R., & Lovorn, M.
    (2010) Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15(1), 18–39. 10.1016/j.asw.2010.01.003
     https://doi.org/10.1016/j.asw.2010.01.003 [Google Scholar]
  53. Schoonen, R.
    (2005) Generalizability of writing scores. An application of structural equation modeling. Language Testing, 22(1) 1–30. 10.1191/0265532205lt295oa
     https://doi.org/10.1191/0265532205lt295oa [Google Scholar]
  54. Timpe, V.
    (2013) Assessing intercultural communicative competence. The dependence of receptive sociopragmatic competence and discourse competence on learning opportunities and input. Peter Lang.
     [Google Scholar]
  55. Timpe-Laughlin, V.
    (2018) Pragmatics in task-based language assessment. Opportunities and challenges. InN. Taguchi, & Y-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 288–304). John Benjamins. 10.1075/tblt.10.12tim
     https://doi.org/10.1075/tblt.10.12tim [Google Scholar]
  56. Upshur, J. A., & Turner, C. E.
    (1995) Constructing rating scales for second language tests. ELT Journal, 49 (1), 3–12. 10.1093/elt/49.1.3
     https://doi.org/10.1093/elt/49.1.3 [Google Scholar]
  57. Vasylets, O., Gilabert, R., & Manchón, R. M.
    (2019) Differential contribution of oral and written modes to lexical, syntactic and propositional complexity in L2 performance in instructed contexts. Instructed Second Language Acquisition, 3(2), 206–227. 10.1558/isla.38289
     https://doi.org/10.1558/isla.38289 [Google Scholar]
  58. Weigle, S.
    (2002) Assessing writing. Cambridge University Press. 10.1017/CBO9780511732997
     https://doi.org/10.1017/CBO9780511732997 [Google Scholar]
  59. Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y.
    (1998) Second language development in writing: Measures of fluency, accuracy, and complexity. University of Hawai’i Press.
     [Google Scholar]
/content/journals/10.1075/task.00013.kui
Loading
Measurement of functional adequacy in different learning contexts
task 2, 8 (2022); https://doi.org/10.1075/task.00013.kui
/content/journals/10.1075/task.00013.kui
/content/journals/10.1075/task.00013.kui
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): applicability; functional adequacy (FA); rating scale; reliability; task-based language assessment (TBLA); validity

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error