1887
  1. Home
  2. e-Journals
  3. Applied Pragmatics
  4. Volume 7, Issue 2
  5. Article
Volume 7, Issue 2
  • ISSN 2589-109X
  • E-ISSN: 2589-1103
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study reports findings from a mixed-effects analysis on item- and person-levels for five intact classes of learners following a semester-long pragmatics instruction. Data represent multiple-choice responses on an instrument assessing pragmatic awareness from learners at three levels of formal Japanese education (lower- to post-secondary). Instructional interventions differed in the degree of explicitness for both pragmalinguistics and sociopragmatics. Ten speech acts represented pragmalinguistic forms (apologies, complaints, compliments, farewells, greetings, introductions, invitations, offers, requests, and suggestions), with the number of speech acts taught operationalized as part of the explicitness. Twelve relationship status categories represented the sociopragmatic feature (acquaintances, best friends, boss/employee, coworkers, customer/service person, family, friends, girlfriend/boyfriend, wife/husband, professional relationship, neighbors, and strangers). Item-level results reveal a significant effect regarding learner accuracy on items measuring sociopragmatic content relative to pragmalinguistic content. Person-level results reveal that increased explicitness accounted for differences between intervention groups. However, the highest-performing group did not receive the most explicit instruction. The discussion includes an argument for greater use of video-based content in pragmatic instruction and assessment practices and the value of mixed-effects models when analyzing longitudinal classroom research across multiple sites.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/ap.22005.ryl
2025-11-04
2025-11-17

  • From This Site

    /content/journals/10.1075/ap.22005.ryl
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Alcón-Soler, E.
    (2005) Does instruction work for learning pragmatics in the EFL context?System, 331, 417–435. 10.1016/j.system.2005.06.005
     https://doi.org/10.1016/j.system.2005.06.005 [Google Scholar]
  2. Al-Gahtani, S., & Roever, C.
    (2015) The development of requests by L2 learners of modern standard Arabic: A longitudinal and cross-sectional study. Foreign Language Annals, 48(4), 570–583. 10.1111/flan.12157
     https://doi.org/10.1111/flan.12157 [Google Scholar]
  3. Austin, J.
    (1962) How to do things with words. Clarendon Press.
     [Google Scholar]
  4. Bachman, L.
    (1990) Fundamental considerations in language testing. Oxford University Press.
     [Google Scholar]
  5. Bachman, L., & Palmer, A.
    (1996) Language testing in practice. Oxford University Press.
     [Google Scholar]
  6. (2010) Language assessment in practice. Oxford University Press.
     [Google Scholar]
  7. Badjadi, N.
    (2016) A meta-analysis of the effects of instructional tasks on L2 pragmatics comprehension and production. InS. F. Tang & L. Logonnathan (Eds.), Assessment for learning within and beyond the classroom (pp. 241–268). Springer. 10.1007/978‑981‑10‑0908‑2_21
     https://doi.org/10.1007/978-981-10-0908-2_21 [Google Scholar]
  8. Bardovi-Harlig, K., & Shin, S. Y.
    (2014) Expanding traditional testing measures with tasks from L2 pragmatics research. Iranian Journal of Language Testing, 4(1), 26–49.
     [Google Scholar]
  9. Barron, A.
    (2012) Interlanguage pragmatics: From use to acquisition to second language pedagogy. Language Teaching, 45(1). 44–63. 10.1017/S0261444811000462
     https://doi.org/10.1017/S0261444811000462 [Google Scholar]
  10. Blum-Kulka, S., & Olshtain, E.
    (1984) Requests and apologies: A cross-cultural study of speech act realization patterns (CCSARP). Applied Linguistics, 5(3), 196–213. 10.1093/applin/5.3.196
     https://doi.org/10.1093/applin/5.3.196 [Google Scholar]
  11. Brown, P., & Levinson, S.
    (1987) Politeness: Some universals in language use. Cambridge University Press. 10.1017/CBO9780511813085
     https://doi.org/10.1017/CBO9780511813085 [Google Scholar]
  12. Bygate, M.
    (2009) Teaching and testing speaking. InM. Long & C. Doughty (Eds.), The handbook of language teaching (pp. 412–462). Wiley-Blackwell. 10.1002/9781444315783.ch23
     https://doi.org/10.1002/9781444315783.ch23 [Google Scholar]
  13. Byon, A. S.
    (2004) Sociopragmatic analysis of Korean requests: Pedagogical settings. Journal of Pragmatics, 361, 1673–1704. 10.1016/j.pragma.2004.05.003
     https://doi.org/10.1016/j.pragma.2004.05.003 [Google Scholar]
  14. Canale, M.
    (1983) On some dimensions of language proficiency. InJ. Oller, Jr. (Ed.), Issues in language testing research (pp. 333–342). Newbury House.
     [Google Scholar]
  15. Canale, M., & Swain, M.
    (1980) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. 10.1093/applin/1.1.1
     https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  16. Celce-Murcia, M., Dörnyei, Z., & Thurrell, S.
    (1995) Communicative competence: A pedagogically motivated model with content specifications. Issues in Applied Linguistics, 6(2), 5–35. 10.5070/L462005216
     https://doi.org/10.5070/L462005216 [Google Scholar]
  17. Chapelle, C.
    (1994) Are C-tests valid measures for L2 vocabulary research?Second Language Research, 101, 157–187. 10.1177/026765839401000203
     https://doi.org/10.1177/026765839401000203 [Google Scholar]
  18. (1999) Construct definition and validity inquiry in SLA research. InL. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 32–70). Cambridge University Press. 10.1017/CBO9781139524711.004
     https://doi.org/10.1017/CBO9781139524711.004 [Google Scholar]
  19. Cobb, T.
    (n.d.). Range for texts (Version 3) [Computer software]. RetrievedJuly 15, 2021, fromhttps://www.lextutor.ca/cgi-bin/range/texts/index.pl
  20. Cohen, A. D., Salazar-Kleiner, S., & Robinson, P.
    (n.d.). Pragmatics and speech acts. Pragmatics and Speech Acts Bibliography. RetrievedApril 2, 2023, fromhttps://carla.umn.edu/speechacts/bibliography/index.php
     [Google Scholar]
  21. Crystal, D.
    (1997) The Cambridge encyclopedia of language (2nd ed.). Cambridge University Press.
     [Google Scholar]
  22. Cunnings, I.
    (2012) An overview of mixed-effects statistical models for second language researchers. Second Language Research, 28(3), 369–382. 10.1177/0267658312443651
     https://doi.org/10.1177/0267658312443651 [Google Scholar]
  23. DeKeyser, R.
    (2003) Implicit and explicit learning. InC. Doughty & M. Long (Eds.), Handbook of second language acquisition (pp. 313–348). Blackwell. 10.1002/9780470756492.ch11
     https://doi.org/10.1002/9780470756492.ch11 [Google Scholar]
  24. (2007) Introduction: Situating the concept of practice. InR. M. DeKeyser (Ed.), Practice in second language: Perspectives from applied linguistics and cognitive psychology (pp. 1–18). Cambridge University Press. 10.1017/CBO9780511667275.002
     https://doi.org/10.1017/CBO9780511667275.002 [Google Scholar]
  25. DeKeyser, R., & Larson-Hall, J.
    (2005) What does the critical period really mean?InJ. F. Kroll & A. M. D. De Groot (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 88–108). Oxford University Press.
     [Google Scholar]
  26. Fernández Guerra, A. & Martínez Flor, A.
    (2003) Requests in films and in EFL textbooks: A comparison. Elia: Estudios de lingüística inglesa aplicada, 41, 17–34.
     [Google Scholar]
  27. Fisher, W. P., Jr.
    (2007) Rasch measurement and the R statistics environment. Transactions of the Rasch Measurement SIG American Educational Research Association, 21(1), 1095.
     [Google Scholar]
  28. Flowerdew, J.
    (1990) Problems of speech act theory from an applied perspective. Language Learning, 40(1), 79–105. 10.1111/j.1467‑1770.1990.tb00955.x
     https://doi.org/10.1111/j.1467-1770.1990.tb00955.x [Google Scholar]
  29. (2009) Corpora in language teaching. InM. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 327–350). Wiley-Blackwell. 10.1002/9781444315783.ch19
     https://doi.org/10.1002/9781444315783.ch19 [Google Scholar]
  30. Frater, P.
    (2022, May24). Subscription video continues to gain in Japan — Study. Variety. Retrieved fromhttps://variety.com/2022/streaming/asia/subscription-video-japan-1235276063/
     [Google Scholar]
  31. Geis, M. L.
    (1995) Speech acts and conversational interaction. Cambridge University Press. 10.1017/CBO9780511554452
     https://doi.org/10.1017/CBO9780511554452 [Google Scholar]
  32. Golato, A.
    (2003) Studying compliment responses: A comparison of DCTs and naturally occurring talk. Applied Linguistics, 24(1), 90–121. 10.1093/applin/24.1.90
     https://doi.org/10.1093/applin/24.1.90 [Google Scholar]
  33. Grabe, W.
    (2009) Teaching and testing reading. InM. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 441–462). Wiley-Blackwell. 10.1002/9781444315783.ch24
     https://doi.org/10.1002/9781444315783.ch24 [Google Scholar]
  34. Grabowski, K.
    (2013) Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. InS. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 149–171). Palgrave MacMillan. 10.1057/9781137003522_6
     https://doi.org/10.1057/9781137003522_6 [Google Scholar]
  35. Grant, L., & Starks, D.
    (2001) Screening appropriate teaching materials: Closings from textbooks and television soap operas. International Review of Applied Linguistics in Language Teaching, 39(1), 39–50. 10.1515/iral.39.1.39
     https://doi.org/10.1515/iral.39.1.39 [Google Scholar]
  36. Henrichsen, L. E.
    (1984) Sandhi-variation: A filter of input for learners of ESL. Language Learning, 341, 103–126. 10.1111/j.1467‑1770.1984.tb00343.x
     https://doi.org/10.1111/j.1467-1770.1984.tb00343.x [Google Scholar]
  37. Houck, N. R., & Fujimori, J.
    (2010) Teacher, you should lose some weight: Advice giving in English. InD. H. Tatsuki & N. R. Houck (Eds.), Pragmatics: Teaching speech acts (pp. 89–104). TESOL.
     [Google Scholar]
  38. Hudson, T., Detmer, E., & Brown, J. D.
    (1992) A framework for testing cross-cultural pragmatics. University of Hawaiʻi at Mānoa.
     [Google Scholar]
  39. (1995) Developing prototypic measures of cross-cultural pragmatics (Technical Report #7). University of Hawaiʻi, Second Language Teaching and Curriculum Center.
     [Google Scholar]
  40. Hymes, D. H.
    (1972) On communicative competence. InJ. B. Pride & J. Holmes (Eds.), Sociolinguistics. Penguin.
     [Google Scholar]
  41. Ito, Y.
    (2006) The comprehension of English reduced forms by second language learners and its effect on input-intake process. InJ. D. Brown & K. Kondo-Brown (Eds.), Perspectives on teaching connected speech to second language speakers (pp. 67–81). University of Hawaiʻi, National Foreign Language Resource Center.
     [Google Scholar]
  42. Jeon, E. H., & Kaya, T.
    (2006) Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. InJ. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 165–211). John Benjamins. 10.1075/lllt.13.10jeo
     https://doi.org/10.1075/lllt.13.10jeo [Google Scholar]
  43. Kasper, G.
    (2006) Beyond repair: Conversation analysis as an approach to SLA. AILA Review, 19(1), 83–99. 10.1075/aila.19.07kas
     https://doi.org/10.1075/aila.19.07kas [Google Scholar]
  44. (2007) Pragmatics in second language learning: Current developments. Foreign Languages in China, 111(6), 17–25.
     [Google Scholar]
  45. (2009) Locating cognition in second language interaction and learning: Inside the skull or in public view. International Review of Applied Linguistics in Language Teaching, 471. 11–36. 10.1515/iral.2009.002
     https://doi.org/10.1515/iral.2009.002 [Google Scholar]
  46. Kasper, G., & Dahl, M.
    (1991) Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 131, 215–247. 10.1017/S0272263100009955
     https://doi.org/10.1017/S0272263100009955 [Google Scholar]
  47. Kasper, G., & Rose, K. R.
    (2002) Pragmatic development in a second language. Blackwell.
     [Google Scholar]
  48. Knapp, A.
    (2011) Using English as a lingua franca for (mis-)managing conflict in an international university context: An example from a course in engineering. Journal of Pragmatics, 431, 978–990. 10.1016/j.pragma.2010.08.008
     https://doi.org/10.1016/j.pragma.2010.08.008 [Google Scholar]
  49. Larsen-Freeman, D.
    (2009) Teaching and testing grammar. InM. Long & C. Doughty (Eds.), Handbook of Language Teaching (pp. 518–542). Wiley-Blackwell. 10.1002/9781444315783.ch27
     https://doi.org/10.1002/9781444315783.ch27 [Google Scholar]
  50. Leech, G.
    (1983) The principles of pragmatics. Longman.
     [Google Scholar]
  51. Li, S.
    (2019) Cognitive approaches in L2 pragmatics research. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 113–127). Routledge. 10.4324/9781351164085‑8
     https://doi.org/10.4324/9781351164085-8 [Google Scholar]
  52. Linacre, J. M.
    (2016) Winsteps® (Version 3.92.0) [Computer software]. Winsteps.com. RetrievedJanuary 1, 2016, fromwww.winsteps.com/
     [Google Scholar]
  53. Linck, J. A., & Cunnings, I.
    (2015) The utility and application of mixed-effects models in second language research. Language Learning, 65(S1), 185–207. 10.1111/lang.12117
     https://doi.org/10.1111/lang.12117 [Google Scholar]
  54. McNamara, T., & Roever, C.
    (2006) Language testing: The social dimension. Blackwell. www.loc.gov/catdir/toc/ecip0616/2006021660.html
     [Google Scholar]
  55. Messick, S.
    (1996) Validity and washback in language testing. ETS Research Report Series, 1996(1), i–18. 10.1002/j.2333‑8504.1996.tb01695.x
     https://doi.org/10.1002/j.2333-8504.1996.tb01695.x [Google Scholar]
  56. Miura, J.
    (2022, September5). Government sets goal of 100,000 students going abroad by 2027. The Asahi Shimbun. RetrievedFeb 1, 2023, fromhttps://www.asahi.com/ajw/articles/14711306
     [Google Scholar]
  57. Nation, P., & Chung, M.
    (2009) Teaching and testing vocabulary. InM. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 543–559). Wiley-Blackwell. 10.1002/9781444315783.ch28
     https://doi.org/10.1002/9781444315783.ch28 [Google Scholar]
  58. Nguyen, T. T. M.
    (2019) Data collection methods in L2 pragmatics research: An overview. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 195–211). Routledge. 10.4324/9781351164085‑13
     https://doi.org/10.4324/9781351164085-13 [Google Scholar]
  59. Nguyen, T. T. M., Do, T. T. H., Nguyen, A. T., & Pham, T. T. T.
    (2015) Teaching email requests in the academic context: A focus on the role of corrective feedback. Language Awareness, 24(2), 169–195. 10.1080/09658416.2015.1010543
     https://doi.org/10.1080/09658416.2015.1010543 [Google Scholar]
  60. Norris, J., & Ortega, L.
    (2000) Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. 10.1111/0023‑8333.00136
     https://doi.org/10.1111/0023-8333.00136 [Google Scholar]
  61. (2003) Defining and measuring SLA. InC. J. Doughty & M. H. Long (Eds.), The handbook of second language acquisition (pp. 716–761). Blackwell. 10.1002/9780470756492.ch21
     https://doi.org/10.1002/9780470756492.ch21 [Google Scholar]
  62. Plonsky, L., & Gass, S.
    (2011) Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61(2), 325–366. 10.1111/j.1467‑9922.2011.00640.x
     https://doi.org/10.1111/j.1467-9922.2011.00640.x [Google Scholar]
  63. Plonsky, L., & Zhuang, J.
    (2019) A meta-analysis of second language pragmatics instruction. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 287–307). Routledge.
     [Google Scholar]
  64. Polio, C., & Williams, J.
    (2009) Teaching and testing writing. InM. Long & C. Doughty (Eds.), Handbook of language teaching, (pp. 486–517). Wiley-Blackwell. 10.1002/9781444315783.ch26
     https://doi.org/10.1002/9781444315783.ch26 [Google Scholar]
  65. R Core Team
    R Core Team (2016) R: A language and environment for statistical computing. R Foundation for Statistical Computing. www.R-project.org/
     [Google Scholar]
  66. Ren, W.
    (2012) Pragmatic development in Chinese speakers’ L2 English refusals. EUROSLA Yearbook, 121, 63–87. 10.1075/eurosla.12.05ren
     https://doi.org/10.1075/eurosla.12.05ren [Google Scholar]
  67. Roever, C.
    (2011) Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463–481. 10.1177/0265532210394633
     https://doi.org/10.1177/0265532210394633 [Google Scholar]
  68. (2021) Teaching and testing second language pragmatics and interaction: A practical guide (1st ed.). Routledge. 10.4324/9780429260766
     https://doi.org/10.4324/9780429260766 [Google Scholar]
  69. Rose, K.
    (1997) Film in interlanguage pragmatics research. Perspectives, 9(1), 111–144.
     [Google Scholar]
  70. (2005) The effects of instruction in second language pragmatics. System, 33(3), 385–399. 10.1016/j.system.2005.06.003
     https://doi.org/10.1016/j.system.2005.06.003 [Google Scholar]
  71. Rylander, J., Clark, P., & Derrah, R.
    (2013) A video-based method of assessing pragmatic awareness. InS. J. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 65–97). Palgrave Macmillan. 10.1057/9781137003522_3
     https://doi.org/10.1057/9781137003522_3 [Google Scholar]
  72. Rylander, J., Collins, B., Derrah, R., Ferguson, P., & D’Andrea, J.
    (2014) Speech acts: Piloting a pragmatics course in the Japanese context. InN. Sonda & A. Krause (Eds.), JALT2013 conference proceedings. Japan Association for Language Teaching.
     [Google Scholar]
  73. Savignon, S. J.
    (1983) Communicative competence: Theory and classroom practice. Addison-Wesley.
     [Google Scholar]
  74. Schegloff, E.
    (2007) Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press. 10.1017/CBO9780511791208
     https://doi.org/10.1017/CBO9780511791208 [Google Scholar]
  75. Schmidt, R.
    (1993) Consciousness, learning and interlanguage pragmatics. InG. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 21–42). Oxford University Press. 10.1093/oso/9780195066029.003.0002
     https://doi.org/10.1093/oso/9780195066029.003.0002 [Google Scholar]
  76. (2010) Attention, awareness, and individual differences in language learning. InW. M. Chan, S. Chi, K. N. Cin, J. Istanto, M. Nagami, J. W. Sew, T. Suthiwan, & I. Walker (Eds.), Proceedings of CLaSIC 2010 (pp. 721–737). National University of Singapore, Centre for Language Studies.
     [Google Scholar]
  77. Searle, J. R.
    (1969) Speech acts. Cambridge University Press. 10.1017/CBO9781139173438
     https://doi.org/10.1017/CBO9781139173438 [Google Scholar]
  78. Taguchi, N.
    (2019) Second language acquisition and pragmatics: An overview. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 1–14). Routledge.
     [Google Scholar]
  79. Taguchi, N., & Roever, C.
    (2017) Second language pragmatics. Oxford University Press.
     [Google Scholar]
  80. Takahashi, S.
    (2010) Assessing learnability in second language pragmatics. InA. Trosborg (Ed.), Handbook of pragmatics (Vol.71, pp. 391–421). Mouton de Gruyter.
     [Google Scholar]
  81. Takimoto, M.
    (2006) The effects of explicit feedback on the development of pragmatic proficiency. Language Teaching Research, 101, 393–417. 10.1191/1362168806lr198oa
     https://doi.org/10.1191/1362168806lr198oa [Google Scholar]
  82. (2009) The effects of input-based tasks on the development of learners’ pragmatic proficiency. Applied Linguistics, 30(1), 1–25. 10.1093/applin/amm049
     https://doi.org/10.1093/applin/amm049 [Google Scholar]
  83. (2012) Assessing the effects of identical task repetition and task-type repetition on learners’ recognition and production of second language request downgraders. Intercultural Pragmatics, 9(1), 71–96. 10.1515/ip‑2012‑0004
     https://doi.org/10.1515/ip-2012-0004 [Google Scholar]
  84. Thomas, J.
    (1983) Cross-cultural pragmatic failure. Applied Linguistics, 4(2), 91–112. 10.1093/applin/4.2.91
     https://doi.org/10.1093/applin/4.2.91 [Google Scholar]
  85. Vandergrift, L., & Goh, C.
    (2009) Teaching and testing listening comprehension. InM. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 395–411). Wiley-Blackwell. 10.1002/9781444315783.ch22
     https://doi.org/10.1002/9781444315783.ch22 [Google Scholar]
  86. Wall, D.
    (2012) Washback. InG. Flucher & F. Davidson (Eds.), The Routledge handbook of language testing (1st ed., pp. 79–92). Routledge.
     [Google Scholar]
  87. Walters, A.
    (2009) Advances in materials design. InM. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 311–326). Wiley-Blackwell. 10.1002/9781444315783.ch18
     https://doi.org/10.1002/9781444315783.ch18 [Google Scholar]
  88. Webb, S., & Rodgers, M.
    (2009) Vocabulary demands of television programs. Language Learning, 59(2), 335–366. 10.1111/j.1467‑9922.2009.00509.x
     https://doi.org/10.1111/j.1467-9922.2009.00509.x [Google Scholar]
  89. Woodfield, H.
    (2008) Problematising discourse completion tasks: Voices from verbal report. Evaluation & Research in Education, 21(1), 43–69. 10.2167/eri413.0
     https://doi.org/10.2167/eri413.0 [Google Scholar]
/content/journals/10.1075/ap.22005.ryl
Loading
Learner gains from pragmatics instruction across contexts
Applied Pragmatics 7, 192 (2025); https://doi.org/10.1075/ap.22005.ryl
/content/journals/10.1075/ap.22005.ryl
/content/journals/10.1075/ap.22005.ryl
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): assessing pragmatic awareness; interlanguage pragmatics; mixed-effects models

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error