1887
  1. Home
  2. e-Journals
  3. Applied Pragmatics
  4. Volume 2, Issue 1
  5. Article
Volume 2, Issue 1
  • ISSN 2589-109X
  • E-ISSN: 2589-1103
USD
Buy:$35.00 + Taxes

Abstract

Discourse Completion Tasks (DCTs) have been one of the most popular tools in pragmatics research. Yet, many have criticized DCTs for their lack of authenticity (e.g., Culpeper, Mackey, & Taguchi, 2018Nguyen, 2019). We propose that corpora can serve as resources in designing and evaluating DCTs. We created a DCT using advice-seeking prompts from the Q+A corpus (Baker & Egbert, 2016). Then, we administered the DCT to 33 participants. We evaluated the DCT by (1) comparing the linguistic form and the semantic content of the participants’ DCT responses (i.e., advice-giving expressions) with authentic data from the corpus; and (2) interviewing the participants about the instrument quality. Chi-square tests between DCT data and corpus data revealed no significant differences in advice-giving expressions in terms of both the overall level of directness ( [2,  = 660] = 6.94,  = .03,  = .10) and linguistic realization ( [8,  = 660] = 17.75,  = .02,  = .16), and showed a significant difference but small effect size in terms of semantic content ( [6,  = 512] = 30.35,  < .01,  = .24). Taken together with the interview data, our findings indicate that corpora are useful in designing DCTs.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/ap.19009.has
2020-03-10
2026-07-16

  • From This Site

    /content/journals/10.1075/ap.19009.has
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Aijmer, K., & Rühlemann, C.
    (Eds.) (2015) Corpus pragmatics. Cambridge: Cambridge University Press. 10.1017/CBO9781139057493
     https://doi.org/10.1017/CBO9781139057493 [Google Scholar]
  2. Alcón, E., & Safont, P.
    (2001) Occurrence of exhortative speech acts in ELT materials and natural speech data: A focus on request, suggestion and advice realization strategies. Studies in English Language and Linguistics, 3, 5–22.
     [Google Scholar]
  3. Bachman, L. F.
    (2002) Some reflections on task-based language performance assessment. Language Testing, 19(4), 453–476. 10.1191/0265532202lt240oa
     https://doi.org/10.1191/0265532202lt240oa [Google Scholar]
  4. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
     [Google Scholar]
  5. Baker, P., & Egbert, J.
    (Eds.) (2016) Triangulating methodological approaches in corpus linguistic research. New York, NY: Routledge. 10.4324/9781315724812
     https://doi.org/10.4324/9781315724812 [Google Scholar]
  6. Bardovi-Harlig, K.
    (2018) Matching modality in L2 pragmatics research design. System, 75, 13–22. 10.1016/j.system.2018.03.007
     https://doi.org/10.1016/j.system.2018.03.007 [Google Scholar]
  7. Bardovi-Harlig, K., Mossman, S., & Su, Y.
    (2017) The effect of corpus-based instruction on pragmatic routines. Language Learning & Technology, 21(3), 76–103.
     [Google Scholar]
  8. Bardovi-Harlig, K., Mossman, S., & Vellenga, H. E.
    (2015) Developing corpus-based materials to teach pragmatic routines. TESOL Journal, 6(3), 499–526. 10.1002/tesj.177
     https://doi.org/10.1002/tesj.177 [Google Scholar]
  9. Beebe, L. M., & Cummings, M. C.
    (1996) Natural speech act data versus written questionnaire data: How data collection method affects speech act performance. InS. Gass & J. Neu (Eds.), Speech acts across cultures: Challenges to communication in a second language (pp.65–86). Berlin: Mouton de Gruyter.
     [Google Scholar]
  10. Biber, D., & Conrad, S.
    (2019) Register, genre, and style. Cambridge: Cambridge University Press. 10.1017/9781108686136
     https://doi.org/10.1017/9781108686136 [Google Scholar]
  11. Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., … Urzua, A.
    (2004) Representing language use in the university: Analysis of the TOEFL 2000 spoken and written academic language corpus. (ETS TOEFL Monograph Series, MS-25). Princeton, NJ: Educational Testing Service.
     [Google Scholar]
  12. Billmyer, K., & Varghese, M.
    (2000) Investigating instrument-based pragmatic variability: effects of enhancing discourse completion tests. Applied Linguistics, 21(4), 517–552. 10.1093/applin/21.4.517
     https://doi.org/10.1093/applin/21.4.517 [Google Scholar]
  13. Blum-Kulka, S.
    (1980) Learning to say what you mean in a second language: A study of the speech act performance of learners of Hebrew as a second language. Applied Linguistics, 3, 29–59. 10.1093/applin/3.1.29
     https://doi.org/10.1093/applin/3.1.29 [Google Scholar]
  14. Boyatzis, R. E.
    (1998) Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage.
     [Google Scholar]
  15. Braun, V., & Clarke, V.
    (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. 10.1191/1478088706qp063oa
     https://doi.org/10.1191/1478088706qp063oa [Google Scholar]
  16. Brown, P., & Levinson, S. C.
    (1987) Politeness: Some universals in language use. Cambridge: Cambridge University Press. 10.1017/CBO9780511813085
     https://doi.org/10.1017/CBO9780511813085 [Google Scholar]
  17. Canli, Z., & Canli, B.
    (2013) Keep calm and say sorry!: The use of apologies by EFL teachers in Turkish and English. Educational Process: International Journal, 2(1), 36–46. 10.12973/edupij.2013.212.3
     https://doi.org/10.12973/edupij.2013.212.3 [Google Scholar]
  18. Chapelle, C. A.
    (2012) Validity argument for language assessment: The framework is simple…. Language Testing, 29(1), 19–27. 10.1177/0265532211417211
     https://doi.org/10.1177/0265532211417211 [Google Scholar]
  19. Cohen, A. D., & Olshtain, E.
    (1994) Researching the production of second language speech acts. InE. T. Tarone, S. M. Gass, & A. D. Cohen (Eds.), Research methodology in second language acquisition (pp.143–156). Hillsdale, NJ: Lawrence Erlbaum Associates.
     [Google Scholar]
  20. Culpeper, J., Mackey, A., & Taguchi, N.
    (2018) Second language pragmatics: From theory to methods. New York, NY: Routledge. 10.4324/9781315692388
     https://doi.org/10.4324/9781315692388 [Google Scholar]
  21. Cutrona, C. E. & Suhr, J. A.
    (1994) Social support communication in the context of marriage: An analysis of couples’ supportive interactions. InB. R. Burleson, T. L. Albrecht, & I. G. Sarason (Eds.), Communication of social support: Messages, interactions, relationships, and community (pp.113–135). Thousand Oaks, CA: Sage.
     [Google Scholar]
  22. DeCapua, A., & Dunham, J. F.
    (2007) The pragmatics of advice giving: Cross-cultural perspectives. Intercultural Pragmatics, 4(3), 319–342. 10.1515/IP.2007.016
     https://doi.org/10.1515/IP.2007.016 [Google Scholar]
  23. Eslami-Rasekh, Z.
    (2005) Raising the pragmatic awareness of language learners. ELT Journal, 59(3), 199–208. 10.1093/elt/cci039
     https://doi.org/10.1093/elt/cci039 [Google Scholar]
  24. Flöck, I., & Geluykens, R.
    (2015) Speech acts in corpus pragmatics: A quantitative contrastive study of directives in spontaneous and elicited discourse. InJ. Romero-Trillo (Ed.), Yearbook of corpus linguistics and pragmatics 2015: Current approaches to discourse and translation studies (pp.7–37). Cham: Springer. 10.1007/978‑3‑319‑17948‑3_2
     https://doi.org/10.1007/978-3-319-17948-3_2 [Google Scholar]
  25. Golato, A.
    (2003) Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied Linguistics, 24(1), 90–121. 10.1093/applin/24.1.90
     https://doi.org/10.1093/applin/24.1.90 [Google Scholar]
  26. Goldsmith, D. J.
    (2004) Communicating social support. New York, NY: Cambridge University Press. 10.1017/CBO9780511606984
     https://doi.org/10.1017/CBO9780511606984 [Google Scholar]
  27. Grabowski, K. C.
    (2007) Reconsidering the measurement of pragmatic knowledge using a reciprocal written test format. Columbia University Working Papers in TESOL & Applied Linguistics, 7(1), 1–48.
     [Google Scholar]
  28. Halenko, N., & Jones, C.
    (2011) Teaching pragmatic awareness of spoken requests to Chinese EAP learners in the UK: Is explicit instruction effective?System, 39(2), 240–250. 10.1016/j.system.2011.05.003
     https://doi.org/10.1016/j.system.2011.05.003 [Google Scholar]
  29. Hartford, B. S., & Bardovi-Harlig, K.
    (1992) Experimental and observational data in the study of interlanguage pragmatics. Pragmatics and Language Learning, 3, 33–52.
     [Google Scholar]
  30. Hinkel, E.
    (1997) Appropriateness of advice: DCT and multiple choice data. Applied Linguistics, 18(1), 1–26. 10.1093/applin/18.1.1
     https://doi.org/10.1093/applin/18.1.1 [Google Scholar]
  31. Hong, C. Y., & Shih, S. C.
    (2013) Proficiency and complaints: Analyses of production and perceptions. Intergrams, 14(1), 1–20.
     [Google Scholar]
  32. Johnston, B., Kasper, G., & Ross, S.
    (1998) Effect of rejoinders in production questionnaires. Applied Linguistics, 19(2), 157–182. 10.1093/applin/19.2.157
     https://doi.org/10.1093/applin/19.2.157 [Google Scholar]
  33. Kasper, G., & Dahl, M.
    (1991) Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 13(2), 215–247. 10.1017/S0272263100009955
     https://doi.org/10.1017/S0272263100009955 [Google Scholar]
  34. Kouper, I.
    (2010) The pragmatics of peer advice in a Live Journal community. Language@ internet, 7, article 1.
     [Google Scholar]
  35. Labben, A.
    (2016) Reconsidering the development of the discourse completion test in interlanguage pragmatics. Pragmatics: Quarterly Publication of the International Pragmatics Association (IPrA), 26(1), 69–91. 10.1075/prag.26.1.04lab
     https://doi.org/10.1075/prag.26.1.04lab [Google Scholar]
  36. Martínez-Flor, A. M.
    (2003) Non-native speakers’ production of advice acts: The effects of proficiency. Revista Española de Lingüística Aplicada, 16, 139–153.
     [Google Scholar]
  37. McHugh, M. L.
    (2013) The chi-square test of independence. Biochemia Medica, 23(2), 143–149. 10.11613/BM.2013.018
     https://doi.org/10.11613/BM.2013.018 [Google Scholar]
  38. Nguyen, T. T. M.
    (2019) Data collection methods in L2 pragmatics research: An overview. InN. Taguchi (Ed.), The Routledge handbook of SLA and pragmatics (pp.195–211). New York, NY: Routledge.
     [Google Scholar]
  39. Parvaresh, V., & Tavakoli, M.
    (2009) Discourse completion tasks as elicitation tools: How convergent are they. The Social Sciences, 4(4), 366–373.
     [Google Scholar]
  40. Patton, M. Q.
    (1990) Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.
     [Google Scholar]
  41. Plonsky, L., & Oswald, F. L.
    (2014) How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. 10.1111/lang.12079
     https://doi.org/10.1111/lang.12079 [Google Scholar]
  42. R Core Team
    R Core Team (2013) R: A language and environment for statistical computing (Version 3.4.3) [Computer software]. Vienna, Austria: R Foundation for Statistical Computing.
     [Google Scholar]
  43. Schauer, G. A., & Adolphs, S.
    (2006) Expressions of gratitude in corpus and DCT data: Vocabulary, formulaic sequences, and pedagogy. System, 34(1), 119-134.
     [Google Scholar]
  44. Staples, S., & Fernández, J.
    (2018) Corpus linguistics approaches to L2 pragmatics. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (241–254). Routledge: London.
     [Google Scholar]
  45. Woodfield, H.
    (2008) Problematising discourse completion tasks: Voices from verbal report. Evaluation & Research in Education, 21(1), 43–69. 10.2167/eri413.0
     https://doi.org/10.2167/eri413.0 [Google Scholar]
  46. Yuan, Y.
    (2001) An inquiry into empirical pragmatics data-gathering methods: Written DCTs, oral DCTs, field notes, and natural conversations. Journal of Pragmatics, 33(2), 271–292. 10.1016/S0378‑2166(00)00031‑X
     https://doi.org/10.1016/S0378-2166(00)00031-X [Google Scholar]
  47. (2002) Compliments and compliment responses in Kunming Chinese. Pragmatics: Quarterly Publication of the International Pragmatics Association (IPrA), 12(2), 183–226. 10.1075/prag.12.2.04yua
     https://doi.org/10.1075/prag.12.2.04yua [Google Scholar]
/content/journals/10.1075/ap.19009.has
Loading
Using a corpus in creating and evaluating a DCT
Applied Pragmatics 2, 80 (2020); https://doi.org/10.1075/ap.19009.has
/content/journals/10.1075/ap.19009.has
/content/journals/10.1075/ap.19009.has
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): advice-giving; authenticity; corpus-based instrument; discourse completion task; pragmatics; technology-mediated communication

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error