1887
  1. Home
  2. e-Journals
  3. Journal of Second Language Studies
  4. Volume 6, Issue 1
  5. Article
Volume 6, Issue 1
  • ISSN 2542-3835
  • E-ISSN: 2542-3843
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Rating scales developed to measure interactional competence (IC) are mainly data-driven which can incur unaffordable costs for assessment practitioners with limited resources, such as universities using this test for placement purposes. A cheaper alternative is to use models proposed in the literature to develop such rating scales. This mixed-methods study evaluates a contextualized analytic IC rating scale developed based on Galaczi and Taylor’s (2018) metaphorical conceptualization of IC. A many-facet Rasch analysis indicated this rating scale could separate examinees into three levels of ability, which was lower than the expected four levels. To explore this finding, raters were interviewed about their experiences of using this rating scale. The findings show the potential and limitations of developing contextualized IC ratings scales based on literature and possible avenues for future research. They also suggest a direct interaction between assessment and instructional practices of IC with implications for language teaching.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/jsls.22003.nei
2023-01-10
2024-12-12

  • From This Site

    /content/journals/10.1075/jsls.22003.nei
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Bachman, L. F.
    (2004) Fundamental considerations in language testing (7. impr). Oxford Univ. Press.
     [Google Scholar]
  2. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice. Oxford University Press.
     [Google Scholar]
  3. Bonk, W. J., & Ockey, G. J.
    (2003) A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, 20(1), 89–110. 10.1191/0265532203lt245oa
     https://doi.org/10.1191/0265532203lt245oa [Google Scholar]
  4. Borger, L.
    (2019) Assessing interactional skills in a paired speaking test. Apples – Journal of Applied Language Studies, 13(1), 151–174. 10.17011/apples/urn.201903011694
     https://doi.org/10.17011/apples/urn.201903011694 [Google Scholar]
  5. Canale, M.
    (1983) From communicative competence to communicative langauge pedagogy. InJ. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp.2–28). Longman.
     [Google Scholar]
  6. Canale, M., & Swain, M.
    (1980) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. 10.1093/applin/1.1.1
     https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  7. Creswell, J. W.
    (2010) Mapping th edeveloping landscape of mixed methods research. InA. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp.45–68). SAGE Publications. 10.4135/9781506335193.n2
     https://doi.org/10.4135/9781506335193.n2 [Google Scholar]
  8. Dörnyei, Z.
    (2007) Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford University Press.
     [Google Scholar]
  9. Ducasse, A. M., & Brown, A.
    (2009) Assessing paired orals: Raters’ orientation to interaction. Language Testing, 26(3), 423–443. 10.1177/0265532209104669
     https://doi.org/10.1177/0265532209104669 [Google Scholar]
  10. Eckes, T.
    (2015) Introduction to Many-Facet Rasch Measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang GmbH, Internationaler Verlag der Wissenschaften.
     [Google Scholar]
  11. Ezzy, D.
    (2002) Qualitative analysis: Practice and innovation. Routledge.
     [Google Scholar]
  12. Fulcher, G.
    (2010) Practical language testing. Hodder Education.
     [Google Scholar]
  13. Galaczi, E. D.
    (2014) Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests?Applied Linguistics, 35(5), 553–574. 10.1093/applin/amt017
     https://doi.org/10.1093/applin/amt017 [Google Scholar]
  14. Galaczi, E., & Taylor, L.
    (2018) Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, 15(3), 219–236. 10.1080/15434303.2018.1453816
     https://doi.org/10.1080/15434303.2018.1453816 [Google Scholar]
  15. Galaczi, E., & Taylor, L. B.
    (2021) Measuring interactional competence. InP. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp.338–348). Routledge.
     [Google Scholar]
  16. Hughes, R.
    (2007) Testing the visible: Literate biases in oral language testing. Journal of Applied Linguistics and Professional Practice, 1(3), 295–309. 10.1558/japl.v1.i3.295
     https://doi.org/10.1558/japl.v1.i3.295 [Google Scholar]
  17. Hymes, D.
    (1972) On communicative competence. InJ. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp.53–73). Penguin.
     [Google Scholar]
  18. Ikeda, N.
    (2021) Assessing L2 learners’ pragmatic ability in problem-solving situations at English-medium university. Applied Pragmatics, 3(1), 51–83. 10.1075/ap.19039.ike
     https://doi.org/10.1075/ap.19039.ike [Google Scholar]
  19. Kim, H.
    (2018) What constitutes professional communication in aviation: Is language proficiency enough for testing purposes?Language Testing, 35(3), 403–426. 10.1177/0265532218758127
     https://doi.org/10.1177/0265532218758127 [Google Scholar]
  20. Kramsch, C.
    (1986) From Language Proficiency to Interactional Competence. The Modern Language Journal, 70(4), 366–372. 10.1111/j.1540‑4781.1986.tb05291.x
     https://doi.org/10.1111/j.1540-4781.1986.tb05291.x [Google Scholar]
  21. Lam, D. M. K.
    (2018) What counts as “responding”? Contingency on previous speaker contribution as a feature of interactional competence. Language Testing, 35(3), 377–401. 10.1177/0265532218758126
     https://doi.org/10.1177/0265532218758126 [Google Scholar]
  22. (2021) Don’t Turn a Deaf Ear: A Case for Assessing Interactive Listening. Applied Linguistics, 42(4), 740–764. 10.1093/applin/amaa064
     https://doi.org/10.1093/applin/amaa064 [Google Scholar]
  23. Lantolf, J. P.
    (Ed.) (2000) Sociocultural theory and second language learning. Oxford Univ. Press.
     [Google Scholar]
  24. Lazaraton, A.
    (1996) Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing, 13(2), 151–172. 10.1177/026553229601300202
     https://doi.org/10.1177/026553229601300202 [Google Scholar]
  25. Linacre, J. M.
    (1998) FACETS 3.17. Computer Program. MESA Press.
     [Google Scholar]
  26. (2021) Winsteps® Rasch measurement computer program (5.1.1) [Computer software]. Winsteps.com
     [Google Scholar]
  27. Mackey, A., & Gass, S. M.
    (2010) Second language research: Methodology and design (Repr). Routledge.
     [Google Scholar]
  28. May, L.
    (2011) Interactional Competence in a Paired Speaking Test: Features Salient to Raters. Language Assessment Quarterly, 8(2), 127–145. 10.1080/15434303.2011.565845
     https://doi.org/10.1080/15434303.2011.565845 [Google Scholar]
  29. May, L., Nakatsuhara, F., Lam, D., & Galaczi, E.
    (2020) Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language Testing, 37(2), 165–188. 10.1177/0265532219879044
     https://doi.org/10.1177/0265532219879044 [Google Scholar]
  30. McNamara, T. F., & Roever, C.
    (2006) Language testing: The social dimension. Blackwell Pub.
     [Google Scholar]
  31. Merriam, S. B., & Tisdell, E. J.
    (2016) Qualitative research: A guide to design and implementation (Fourth edition). Jossey-Bass.
     [Google Scholar]
  32. Nakatsuhara, F., May, L., Lam, D., & Galaczi, E.
    (2018) Learning oriented feedback in the development and assessment of interactional competence. Reserach Notes, 701, 1–68.
     [Google Scholar]
  33. Ockey, G. J.
    (2012) Item response theory. InG. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp.336–349).
     [Google Scholar]
  34. Ockey, G. J., & Chukharev-Hudilainen, E.
    (2021) Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. 10.1093/applin/amaa067
     https://doi.org/10.1093/applin/amaa067 [Google Scholar]
  35. Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A.
    (2015) The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62. 10.1177/0265532214538014
     https://doi.org/10.1177/0265532214538014 [Google Scholar]
  36. O’Sullivan, B.
    (2000) Exploring gender and oral proficiency interview performance. System, 28(3), 373–386. 10.1016/S0346‑251X(00)00018‑X
     https://doi.org/10.1016/S0346-251X(00)00018-X [Google Scholar]
  37. O’Sullivan, B., Weir, C. J., & Saville, N.
    (2002) Using observation checklists to validate speaking-test tasks. Language Testing, 19(1), 33–56. 10.1191/0265532202lt219oa
     https://doi.org/10.1191/0265532202lt219oa [Google Scholar]
  38. Pekarek Doehler, S., & Pochon-Berger, E.
    (2011) Developing “methods” for interaction: A cross-sectional study of disagreement sequences in French L2. InJ. K. Hall, J. Hellermann, & S. P. Doehler (Eds.), L2 interactional competence and development (pp.206–243). Multilingual Matters. 10.21832/9781847694072‑010
     https://doi.org/10.21832/9781847694072-010 [Google Scholar]
  39. (2015) The development of L2 interactional competence: Evidence from turn-taking organization, sequence organization, repair organization, an preference organization. InT. Cadierno & S. W. Eskildsen (Eds.), Usage-based perspectives on second language learning (pp.233–268). De Gruyter Mouton. 10.1515/9783110378528‑012
     https://doi.org/10.1515/9783110378528-012 [Google Scholar]
  40. Plough, I., Banerjee, J., & Iwashita, N.
    (2018) Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445. 10.1177/0265532218772325
     https://doi.org/10.1177/0265532218772325 [Google Scholar]
  41. Roever, C., & Ikeda, N.
    (2021) What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing, 026553222110033. 10.1177/02655322211003332
     https://doi.org/10.1177/02655322211003332 [Google Scholar]
  42. Roever, C., & Kasper, G.
    (2018) Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing, 35(3), 331–355. 10.1177/0265532218758128
     https://doi.org/10.1177/0265532218758128 [Google Scholar]
  43. Ross, S.
    (2018) Listener response as a facet of interactional competence. Language Testing, 35(3), 357–375. 10.1177/0265532218758125
     https://doi.org/10.1177/0265532218758125 [Google Scholar]
  44. Sandlund, E., & Greer, T.
    (2020) How do raters understand rubrics for assessing L2 interactional engagement? A comparative study of CA-and non-CA-formulated performance descriptors. Papers in Language Testing and Assessment: An International Journal of the Association for Language Testing and Assessment of Australia and New Zealand, 9(1), 128–163.
     [Google Scholar]
  45. Sandlund, E., Sundqvist, P., & Nyroos, L.
    (2016) Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing: Testing L2 Talk. Language and Linguistics Compass, 10(1), 14–29. 10.1111/lnc3.12174
     https://doi.org/10.1111/lnc3.12174 [Google Scholar]
  46. Vo, S.
    (2021) Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, 12(2). 10.1002/tesj.563
     https://doi.org/10.1002/tesj.563 [Google Scholar]
  47. Wright, B. D.
    (1996) Localy dependency, correlations and prinicpal components. Rasch Measurement Transactions, 101, 509–511.
     [Google Scholar]
  48. Wu, S., & Ma, Z.
    (2020) How is Chinese reading affected by under-specification and over-specification? Evidence from self-paced reading experiments. Journal of Pragmatics, 1551, 213–233. 10.1016/j.pragma.2019.11.006
     https://doi.org/10.1016/j.pragma.2019.11.006 [Google Scholar]
  49. Youn, S. J.
    (2015) Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. 10.1177/0265532214557113
     https://doi.org/10.1177/0265532214557113 [Google Scholar]
  50. Young, R.
    (2008) Language and interaction: An advanced resource book. Routledge.
     [Google Scholar]
/content/journals/10.1075/jsls.22003.nei
Loading
Developing and evaluating a contextualized interactional competence rating scale based on a metaphorical conceptualization
Journal of Second Language Studies 6, 61 (2023); https://doi.org/10.1075/jsls.22003.nei
/content/journals/10.1075/jsls.22003.nei
/content/journals/10.1075/jsls.22003.nei
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): instruction of interactional competence; interactional competence; interactional competence definition evaluation; mixed-methods evaluation; oral proficiency assessment; paired oral discussion; pragmatism; rating scale

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error