1887
image of Developing and evaluating a contextualized interactional competence rating scale based on a metaphorical
conceptualization
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Rating scales developed to measure interactional competence (IC) are mainly data-driven which can incur unaffordable costs for assessment practitioners with limited resources, such as universities using this test for placement purposes. A cheaper alternative is to use models proposed in the literature to develop such rating scales. This mixed-methods study evaluates a contextualized analytic IC rating scale developed based on metaphorical conceptualization of IC. A many-facet Rasch analysis indicated this rating scale could separate examinees into three levels of ability, which was lower than the expected four levels. To explore this finding, raters were interviewed about their experiences of using this rating scale. The findings show the potential and limitations of developing contextualized IC ratings scales based on literature and possible avenues for future research. They also suggest a direct interaction between assessment and instructional practices of IC with implications for language teaching.

Loading

Article metrics loading...

/content/journals/10.1075/jsls.22003.nei
2023-01-10
2023-01-29
Loading full text...

Full text loading...

References

  1. Bachman, L. F.
    (2004) Fundamental considerations in language testing (7. impr). Oxford Univ. Press.
    [Google Scholar]
  2. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice. Oxford University Press.
    [Google Scholar]
  3. Bonk, W. J., & Ockey, G. J.
    (2003) A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, (), –. 10.1191/0265532203lt245oa
    https://doi.org/10.1191/0265532203lt245oa [Google Scholar]
  4. Borger, L.
    (2019) Assessing interactional skills in a paired speaking test. Apples – Journal of Applied Language Studies, (), –. 10.17011/apples/urn.201903011694
    https://doi.org/10.17011/apples/urn.201903011694 [Google Scholar]
  5. Canale, M.
    (1983) From communicative competence to communicative langauge pedagogy. InJ. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp.–). Longman.
    [Google Scholar]
  6. Canale, M., & Swain, M.
    (1980) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, (), –. 10.1093/applin/1.1.1
    https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  7. Creswell, J. W.
    (2010) Mapping th edeveloping landscape of mixed methods research. InA. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp.–). SAGE Publications. 10.4135/9781506335193.n2
    https://doi.org/10.4135/9781506335193.n2 [Google Scholar]
  8. Dörnyei, Z.
    (2007) Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford University Press.
    [Google Scholar]
  9. Ducasse, A. M., & Brown, A.
    (2009) Assessing paired orals: Raters’ orientation to interaction. Language Testing, (), –. 10.1177/0265532209104669
    https://doi.org/10.1177/0265532209104669 [Google Scholar]
  10. Eckes, T.
    (2015) Introduction to Many-Facet Rasch Measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang GmbH, Internationaler Verlag der Wissenschaften.
    [Google Scholar]
  11. Ezzy, D.
    (2002) Qualitative analysis: Practice and innovation. Routledge.
    [Google Scholar]
  12. Fulcher, G.
    (2010) Practical language testing. Hodder Education.
    [Google Scholar]
  13. Galaczi, E. D.
    (2014) Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests?Applied Linguistics, (), –. 10.1093/applin/amt017
    https://doi.org/10.1093/applin/amt017 [Google Scholar]
  14. Galaczi, E., & Taylor, L.
    (2018) Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, (), –. 10.1080/15434303.2018.1453816
    https://doi.org/10.1080/15434303.2018.1453816 [Google Scholar]
  15. Galaczi, E., & Taylor, L. B.
    (2021) Measuring interactional competence. InP. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp.–). Routledge.
    [Google Scholar]
  16. Hughes, R.
    (2007) Testing the visible: Literate biases in oral language testing. Journal of Applied Linguistics and Professional Practice, (), –. 10.1558/japl.v1.i3.295
    https://doi.org/10.1558/japl.v1.i3.295 [Google Scholar]
  17. Hymes, D.
    (1972) On communicative competence. InJ. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp.–). Penguin.
    [Google Scholar]
  18. Ikeda, N.
    (2021) Assessing L2 learners’ pragmatic ability in problem-solving situations at English-medium university. Applied Pragmatics, (), –. 10.1075/ap.19039.ike
    https://doi.org/10.1075/ap.19039.ike [Google Scholar]
  19. Kim, H.
    (2018) What constitutes professional communication in aviation: Is language proficiency enough for testing purposes?Language Testing, (), –. 10.1177/0265532218758127
    https://doi.org/10.1177/0265532218758127 [Google Scholar]
  20. Kramsch, C.
    (1986) From Language Proficiency to Interactional Competence. The Modern Language Journal, (), –. 10.1111/j.1540‑4781.1986.tb05291.x
    https://doi.org/10.1111/j.1540-4781.1986.tb05291.x [Google Scholar]
  21. Lam, D. M. K.
    (2018) What counts as “responding”? Contingency on previous speaker contribution as a feature of interactional competence. Language Testing, (), –. 10.1177/0265532218758126
    https://doi.org/10.1177/0265532218758126 [Google Scholar]
  22. (2021) Don’t Turn a Deaf Ear: A Case for Assessing Interactive Listening. Applied Linguistics, (), –. 10.1093/applin/amaa064
    https://doi.org/10.1093/applin/amaa064 [Google Scholar]
  23. Lantolf, J. P.
    (Ed.) (2000) Sociocultural theory and second language learning. Oxford Univ. Press.
    [Google Scholar]
  24. Lazaraton, A.
    (1996) Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing, (), –. 10.1177/026553229601300202
    https://doi.org/10.1177/026553229601300202 [Google Scholar]
  25. Linacre, J. M.
    (1998) FACETS 3.17. Computer Program. MESA Press.
    [Google Scholar]
  26. (2021) Winsteps® Rasch measurement computer program (5.1.1) [Computer software]. Winsteps.com
    [Google Scholar]
  27. Mackey, A., & Gass, S. M.
    (2010) Second language research: Methodology and design (Repr). Routledge.
    [Google Scholar]
  28. May, L.
    (2011) Interactional Competence in a Paired Speaking Test: Features Salient to Raters. Language Assessment Quarterly, (), –. 10.1080/15434303.2011.565845
    https://doi.org/10.1080/15434303.2011.565845 [Google Scholar]
  29. May, L., Nakatsuhara, F., Lam, D., & Galaczi, E.
    (2020) Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language Testing, (), –. 10.1177/0265532219879044
    https://doi.org/10.1177/0265532219879044 [Google Scholar]
  30. McNamara, T. F., & Roever, C.
    (2006) Language testing: The social dimension. Blackwell Pub.
    [Google Scholar]
  31. Merriam, S. B., & Tisdell, E. J.
    (2016) Qualitative research: A guide to design and implementation (Fourth edition). Jossey-Bass.
    [Google Scholar]
  32. Nakatsuhara, F., May, L., Lam, D., & Galaczi, E.
    (2018) Learning oriented feedback in the development and assessment of interactional competence. Reserach Notes, , –.
    [Google Scholar]
  33. Ockey, G. J.
    (2012) Item response theory. InG. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp.–).
    [Google Scholar]
  34. Ockey, G. J., & Chukharev-Hudilainen, E.
    (2021) Human versus computer partner in the paired oral discussion test. Applied Linguistics, (), –. 10.1093/applin/amaa067
    https://doi.org/10.1093/applin/amaa067 [Google Scholar]
  35. Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A.
    (2015) The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, (), –. 10.1177/0265532214538014
    https://doi.org/10.1177/0265532214538014 [Google Scholar]
  36. O’Sullivan, B.
    (2000) Exploring gender and oral proficiency interview performance. System, (), –. 10.1016/S0346‑251X(00)00018‑X
    https://doi.org/10.1016/S0346-251X(00)00018-X [Google Scholar]
  37. O’Sullivan, B., Weir, C. J., & Saville, N.
    (2002) Using observation checklists to validate speaking-test tasks. Language Testing, (), –. 10.1191/0265532202lt219oa
    https://doi.org/10.1191/0265532202lt219oa [Google Scholar]
  38. Pekarek Doehler, S., & Pochon-Berger, E.
    (2011) Developing “methods” for interaction: A cross-sectional study of disagreement sequences in French L2. InJ. K. Hall, J. Hellermann, & S. P. Doehler (Eds.), L2 interactional competence and development (pp.–). Multilingual Matters. 10.21832/9781847694072‑010
    https://doi.org/10.21832/9781847694072-010 [Google Scholar]
  39. (2015) The development of L2 interactional competence: Evidence from turn-taking organization, sequence organization, repair organization, an preference organization. InT. Cadierno & S. W. Eskildsen (Eds.), Usage-based perspectives on second language learning (pp.–). De Gruyter Mouton. 10.1515/9783110378528‑012
    https://doi.org/10.1515/9783110378528-012 [Google Scholar]
  40. Plough, I., Banerjee, J., & Iwashita, N.
    (2018) Interactional competence: Genie out of the bottle. Language Testing, (), –. 10.1177/0265532218772325
    https://doi.org/10.1177/0265532218772325 [Google Scholar]
  41. Roever, C., & Ikeda, N.
    (2021) What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing, 026553222110033. 10.1177/02655322211003332
    https://doi.org/10.1177/02655322211003332 [Google Scholar]
  42. Roever, C., & Kasper, G.
    (2018) Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing, (), –. 10.1177/0265532218758128
    https://doi.org/10.1177/0265532218758128 [Google Scholar]
  43. Ross, S.
    (2018) Listener response as a facet of interactional competence. Language Testing, (), –. 10.1177/0265532218758125
    https://doi.org/10.1177/0265532218758125 [Google Scholar]
  44. Sandlund, E., & Greer, T.
    (2020) How do raters understand rubrics for assessing L2 interactional engagement? A comparative study of CA-and non-CA-formulated performance descriptors. Papers in Language Testing and Assessment: An International Journal of the Association for Language Testing and Assessment of Australia and New Zealand, (), –.
    [Google Scholar]
  45. Sandlund, E., Sundqvist, P., & Nyroos, L.
    (2016) Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing: Testing L2 Talk. Language and Linguistics Compass, (), –. 10.1111/lnc3.12174
    https://doi.org/10.1111/lnc3.12174 [Google Scholar]
  46. Vo, S.
    (2021) Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, (). 10.1002/tesj.563
    https://doi.org/10.1002/tesj.563 [Google Scholar]
  47. Wright, B. D.
    (1996) Localy dependency, correlations and prinicpal components. Rasch Measurement Transactions, , –.
    [Google Scholar]
  48. Wu, S., & Ma, Z.
    (2020) How is Chinese reading affected by under-specification and over-specification? Evidence from self-paced reading experiments. Journal of Pragmatics, , –. 10.1016/j.pragma.2019.11.006
    https://doi.org/10.1016/j.pragma.2019.11.006 [Google Scholar]
  49. Youn, S. J.
    (2015) Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, (), –. 10.1177/0265532214557113
    https://doi.org/10.1177/0265532214557113 [Google Scholar]
  50. Young, R.
    (2008) Language and interaction: An advanced resource book. Routledge.
    [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/jsls.22003.nei
Loading
/content/journals/10.1075/jsls.22003.nei
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error