1887
Volume 25, Issue 1
  • ISSN 1572-0373
  • E-ISSN: 1572-0381
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Research on interaction in speaking assessment suggests that both verbal and nonverbal interaction are integral parts of the construct of interactional competence (Galaczi & Taylor, 2018; Plough et al., 2018; Young, 2011). However, little has been done to investigate which features significantly contribute to interactional competence scores. This study, therefore, examined which interaction features that raters noticed in individual scripted interview and paired discussion tasks to gain an insight into the interactional competence construct, providing validity evidence for an inclusion of interactional competence in speaking assessment. Sixty-eight student performances were rated based on interaction rating scales. Exploratory factor analysis revealed four factors: nonverbal communication, topic management, interactional management, and interactive listening. Logistic regressions showed that while raters attended to more topic management features in the individual scripted interview task, they noticed more interactional management features in the paired discussion task. Simple regressions showed that nonverbal communication and topic management features predicted interactional competence scores in the individual scripted interview task, whereas nonverbal communication, topic management, interactional management, and interactive listening features were predictors of scores in the paired discussion task. The findings suggest that both nonverbal and verbal interaction features are important in the interactional competence construct with the paired task providing test-takers with more opportunities to demonstrate their interactional ability.

Loading

Article metrics loading...

/content/journals/10.1075/is.00022.vo
2024-06-07
2024-06-19
Loading full text...

Full text loading...

References

  1. Andrich, D.
    (1978) A rating formulation for ordered response categories. Psychometrika, 431, 561–573. 10.1007/BF02293814
    https://doi.org/10.1007/BF02293814 [Google Scholar]
  2. Brooks, L.
    (2009) Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26(3), 342–366. 10.1177/0265532209104666
    https://doi.org/10.1177/0265532209104666 [Google Scholar]
  3. Chapelle, C. A., Enright, M. K., & Jamieson, J.
    (Eds.) (2008) Building a validity argument for the Test of English as a Foreign Language. London, English: Routledge.
    [Google Scholar]
  4. Cohen, J.
    (1988) Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
    [Google Scholar]
  5. Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T.
    (1999) Dictionary of language testing. Cambridge University Press.
    [Google Scholar]
  6. De Winter, J. C., Dodou, D., & Wieringa, P. A.
    (2009) Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44(2), 147–181. 10.1080/00273170902794206
    https://doi.org/10.1080/00273170902794206 [Google Scholar]
  7. Ducasse, A. M.
    (2014) Interaction in paired oral proficiency assessment in Spanish. Frankfurt/Main, Germany: Peter Lang.
    [Google Scholar]
  8. Ducasse, A. M., & Brown, A.
    (2009) Assessing paired orals: Raters’ orientation to interaction. Language Testing, 261, 423–443. 10.1177/0265532209104669
    https://doi.org/10.1177/0265532209104669 [Google Scholar]
  9. Eckes, T.
    (2015) Introduction to Many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang Pub. Inc.
    [Google Scholar]
  10. EPT Oral Communication Test
    EPT Oral Communication Test. Retrieved onFebruary 18, 2020fromhttps://apling.engl.iastate.edu/english-placement-test/
  11. Fabrigar, L. R.
    (2012) Exploratory factor analysis: Understanding statistics. Oxford.
    [Google Scholar]
  12. Galaczi, E. D.
    (2014) IC across proficiency levels: How do learners manage interaction in paired speaking tests?Applied Linguistics, 35(5), 553–574. 10.1093/applin/amt017
    https://doi.org/10.1093/applin/amt017 [Google Scholar]
  13. Galaczi, E. D., & Taylor, L.
    (2018) IC: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236. 10.1080/15434303.2018.1453816
    https://doi.org/10.1080/15434303.2018.1453816 [Google Scholar]
  14. He, L., & Dai, Y.
    (2006) A corpus-based investigation into the validity of the CET–SET group discussion. Language Testing, 231, 370–401. 10.1191/0265532206lt333oa
    https://doi.org/10.1191/0265532206lt333oa [Google Scholar]
  15. Heinze, G., & Schemper, M.
    (2002) A solution to the problem of separation in logistic regression. Statistics in Medicine, 21(16), 2409–2419. 10.1002/sim.1047
    https://doi.org/10.1002/sim.1047 [Google Scholar]
  16. Kline, R.
    (2005) Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford.
    [Google Scholar]
  17. Kramsch, C.
    (1986) From language proficiency to IC. Modern Language Journal, 70(4), 366–372. 10.1111/j.1540‑4781.1986.tb05291.x
    https://doi.org/10.1111/j.1540-4781.1986.tb05291.x [Google Scholar]
  18. Lam, D. M. K.
    (2018) What counts as ‘responding’? Contingency on previous speaker contribution as a feature of IC. Language Testing, 35(3), 377–401. 10.1177/0265532218758126
    https://doi.org/10.1177/0265532218758126 [Google Scholar]
  19. Leech, N. L., Barrett, K. C., & Morgan, G. A.
    (2011) IBM SPSS for intermediate statistics: Use and interpretation (4th ed.). New York, NY: Taylor & Francis.
    [Google Scholar]
  20. Linacre, J. M.
    (2014) Facets (3.71.4) [Computer software]. Winsteps.com. www.winsteps.com/facets.htm
    [Google Scholar]
  21. (1994) Many-facet Rasch measurement (2nd ed.). MESA Press.
    [Google Scholar]
  22. Matsumoto, D.
    (2006) Culture and nonverbal behavior. InV. L. Manusov & M. L. Patterson (Eds.), The SAGE handbook of nonverbal communication (pp. 219–235). Sage Publications, Inc.. 10.4135/9781412976152.n12
    https://doi.org/10.4135/9781412976152.n12 [Google Scholar]
  23. May, L.
    (2011) Interactional competence in a paired speaking test: Features salient to raters. Language Assessment Quarterly, 8(2),127–145. 10.1080/15434303.2011.565845
    https://doi.org/10.1080/15434303.2011.565845 [Google Scholar]
  24. McNeill, D.
    (1985) So you think gestures are nonverbal?Psychological Review, 921, 350–371. 10.1037/0033‑295X.92.3.350
    https://doi.org/10.1037/0033-295X.92.3.350 [Google Scholar]
  25. Neiriz, R.
    (2023) Developing and evaluating a contextualized interactional competence rating scale based on a metaphorical conceptualization: A pragmatic mixed-method approach. Journal of Second Language Studies, 6(1), 61–94. 10.1075/jsls.22003.nei
    https://doi.org/10.1075/jsls.22003.nei [Google Scholar]
  26. Ockey, G. J.
    (2014) The potential of the L2 group oral to elicit discourse with a mutual contingency pattern and afford equal speaking rights in an ESP context. English for Specific Purposes, 351, 17–29. 10.1016/j.esp.2013.11.003
    https://doi.org/10.1016/j.esp.2013.11.003 [Google Scholar]
  27. (2013) Exploratory factor analysis and structural equation modeling. InA. J. Kunnan (Ed.), The companion to language assessment. Vol. III: Evaluation, Methodolody, and Interdisciplinary Themes (pp. 1224–1244, Part101, Chapter 73). Malden, MA: John Wiley & Sons. 10.1002/9781118411360.wbcla114
    https://doi.org/10.1002/9781118411360.wbcla114 [Google Scholar]
  28. Ockey, G. J., & Chukharev-Hudilainen, E.
    (2021) Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. 10.1093/applin/amaa067
    https://doi.org/10.1093/applin/amaa067 [Google Scholar]
  29. Ockey, G. J., & Li, Z.
    (2015) New and not so new methods for assessing oral communication. Language Value, 71, 1–21. 10.6035/LANGUAGEV.2015.7.2
    https://doi.org/10.6035/LANGUAGEV.2015.7.2 [Google Scholar]
  30. Ockey, G. J., & Wagner, E.
    (2018) Assessing L2 listening: Moving towards authenticity, vol. 50, John Benjamins, Philadelphia, PA. 10.1075/lllt.50
    https://doi.org/10.1075/lllt.50 [Google Scholar]
  31. O’Sullivan, B., Weir, C. J., & Saville, N.
    (2002) Using observation checklists to validate speaking test tasks. Language Testing, 191, 33–56. 10.1191/0265532202lt219oa
    https://doi.org/10.1191/0265532202lt219oa [Google Scholar]
  32. Pedhazur, E. J.
    (1997) Multiple regression in behavioral research: Explanation and prediction. Belmont, Ca.: Wadsworth.
    [Google Scholar]
  33. Plough, I., Banerjee, J., & Iwashita, N.
    (2018) Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445. 10.1177/0265532218772325
    https://doi.org/10.1177/0265532218772325 [Google Scholar]
  34. Roever, C., & Kasper, G.
    (2018) Speaking in turns and sequences: IC as a target construct in testing speaking. Language Testing, 35(2), 331–355. 10.1177/0265532218758128
    https://doi.org/10.1177/0265532218758128 [Google Scholar]
  35. Tabachnick, B. G., & Fidell, L. S.
    (2001) Using multivariate statistics. 4th Edition, Allyn and Bacon, Boston.
    [Google Scholar]
  36. Van Lier, L.
    (1989) Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 231, 489–508. 10.2307/3586922
    https://doi.org/10.2307/3586922 [Google Scholar]
  37. Van Moere, A.
    (2007) Group oral test: How does task affect candidate performance and test score? (Unpublished Ph.D. Dissertation). The University of Lancaster.
  38. Vo, S.
    (2021) Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, 12(2), 1–18. 10.1002/tesj.563
    https://doi.org/10.1002/tesj.563 [Google Scholar]
  39. Wright, B. D., & Linacre, J. M.
    (1994) Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370. www.rasch.org/rmt/rmt83b.htm
    [Google Scholar]
  40. Youn, S. J.
    (2020) Interactional features of L2 pragmatic interaction in role-play speaking assessment. TESOL Quarterly, 54(1), 201–233. 10.1002/tesq.542
    https://doi.org/10.1002/tesq.542 [Google Scholar]
  41. (2015) Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. 10.1177/0265532214557113
    https://doi.org/10.1177/0265532214557113 [Google Scholar]
  42. Young, R.
    (2011) Interactional competence in language learning, teaching, and testing. InE. Hinkel (Ed.), Handbook of Research in Second Language Teaching and Learning (pp. 426–443). London, UK & New York, NY: Routledge.
    [Google Scholar]
  43. Yuki, M., Maddux, W. W., & Masuda, T.
    (2007) Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303–311. 10.1016/j.jesp.2006.02.004
    https://doi.org/10.1016/j.jesp.2006.02.004 [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/is.00022.vo
Loading
/content/journals/10.1075/is.00022.vo
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error