1887
  1. Home
  2. e-Journals
  3. Applied Pragmatics
  4. Volume 2, Issue 1
  5. Article
Volume 2, Issue 1
  • ISSN 2589-109X
  • E-ISSN: 2589-1103
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Discourse Completion Tasks (DCTs) have been one of the most popular tools in pragmatics research. Yet, many have criticized DCTs for their lack of authenticity (e.g., Culpeper, Mackey, & Taguchi, 2018Nguyen, 2019). We propose that corpora can serve as resources in designing and evaluating DCTs. We created a DCT using advice-seeking prompts from the Q+A corpus (Baker & Egbert, 2016). Then, we administered the DCT to 33 participants. We evaluated the DCT by (1) comparing the linguistic form and the semantic content of the participants’ DCT responses (i.e., advice-giving expressions) with authentic data from the corpus; and (2) interviewing the participants about the instrument quality. Chi-square tests between DCT data and corpus data revealed no significant differences in advice-giving expressions in terms of both the overall level of directness ( [2,  = 660] = 6.94,  = .03,  = .10) and linguistic realization ( [8,  = 660] = 17.75,  = .02,  = .16), and showed a significant difference but small effect size in terms of semantic content ( [6,  = 512] = 30.35,  < .01,  = .24). Taken together with the interview data, our findings indicate that corpora are useful in designing DCTs.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/ap.19009.has
2020-03-10
2023-09-29

  • From This Site

    /content/journals/10.1075/ap.19009.has
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Aijmer, K., & Rühlemann, C.
    (Eds.) (2015) Corpus pragmatics. Cambridge: Cambridge University Press. 10.1017/CBO9781139057493
     https://doi.org/10.1017/CBO9781139057493 [Google Scholar]
  2. Alcón, E., & Safont, P.
    (2001) Occurrence of exhortative speech acts in ELT materials and natural speech data: A focus on request, suggestion and advice realization strategies. Studies in English Language and Linguistics, 3, 5–22.
     [Google Scholar]
  3. Bachman, L. F.
    (2002) Some reflections on task-based language performance assessment. Language Testing, 19(4), 453–476. 10.1191/0265532202lt240oa
     https://doi.org/10.1191/0265532202lt240oa [Google Scholar]
  4. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
     [Google Scholar]
  5. Baker, P., & Egbert, J.
    (Eds.) (2016) Triangulating methodological approaches in corpus linguistic research. New York, NY: Routledge. 10.4324/9781315724812
     https://doi.org/10.4324/9781315724812 [Google Scholar]
  6. Bardovi-Harlig, K.
    (2018) Matching modality in L2 pragmatics research design. System, 75, 13–22. 10.1016/j.system.2018.03.007
     https://doi.org/10.1016/j.system.2018.03.007 [Google Scholar]
  7. Bardovi-Harlig, K., Mossman, S., & Su, Y.
    (2017) The effect of corpus-based instruction on pragmatic routines. Language Learning & Technology, 21(3), 76–103.
     [Google Scholar]
  8. Bardovi-Harlig, K., Mossman, S., & Vellenga, H. E.
    (2015) Developing corpus-based materials to teach pragmatic routines. TESOL Journal, 6(3), 499–526. 10.1002/tesj.177
     https://doi.org/10.1002/tesj.177 [Google Scholar]
  9. Beebe, L. M., & Cummings, M. C.
    (1996) Natural speech act data versus written questionnaire data: How data collection method affects speech act performance. InS. Gass & J. Neu (Eds.), Speech acts across cultures: Challenges to communication in a second language (pp.65–86). Berlin: Mouton de Gruyter.
     [Google Scholar]
  10. Biber, D., & Conrad, S.
    (2019) Register, genre, and style. Cambridge: Cambridge University Press. 10.1017/9781108686136
     https://doi.org/10.1017/9781108686136 [Google Scholar]
  11. Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., … Urzua, A.
    (2004) Representing language use in the university: Analysis of the TOEFL 2000 spoken and written academic language corpus. (ETS TOEFL Monograph Series, MS-25). Princeton, NJ: Educational Testing Service.
     [Google Scholar]
  12. Billmyer, K., & Varghese, M.
    (2000) Investigating instrument-based pragmatic variability: effects of enhancing discourse completion tests. Applied Linguistics, 21(4), 517–552. 10.1093/applin/21.4.517
     https://doi.org/10.1093/applin/21.4.517 [Google Scholar]
  13. Blum-Kulka, S.
    (1980) Learning to say what you mean in a second language: A study of the speech act performance of learners of Hebrew as a second language. Applied Linguistics, 3, 29–59. 10.1093/applin/3.1.29
     https://doi.org/10.1093/applin/3.1.29 [Google Scholar]
  14. Boyatzis, R. E.
    (1998) Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage.
     [Google Scholar]
  15. Braun, V., & Clarke, V.
    (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. 10.1191/1478088706qp063oa
     https://doi.org/10.1191/1478088706qp063oa [Google Scholar]
  16. Brown, P., & Levinson, S. C.
    (1987) Politeness: Some universals in language use. Cambridge: Cambridge University Press. 10.1017/CBO9780511813085
     https://doi.org/10.1017/CBO9780511813085 [Google Scholar]
  17. Canli, Z., & Canli, B.
    (2013) Keep calm and say sorry!: The use of apologies by EFL teachers in Turkish and English. Educational Process: International Journal, 2(1), 36–46. 10.12973/edupij.2013.212.3
     https://doi.org/10.12973/edupij.2013.212.3 [Google Scholar]
  18. Chapelle, C. A.
    (2012) Validity argument for language assessment: The framework is simple…. Language Testing, 29(1), 19–27. 10.1177/0265532211417211
     https://doi.org/10.1177/0265532211417211 [Google Scholar]
  19. Cohen, A. D., & Olshtain, E.
    (1994) Researching the production of second language speech acts. InE. T. Tarone, S. M. Gass, & A. D. Cohen (Eds.), Research methodology in second language acquisition (pp.143–156). Hillsdale, NJ: Lawrence Erlbaum Associates.
     [Google Scholar]
  20. Culpeper, J., Mackey, A., & Taguchi, N.
    (2018) Second language pragmatics: From theory to methods. New York, NY: Routledge. 10.4324/9781315692388
     https://doi.org/10.4324/9781315692388 [Google Scholar]
  21. Cutrona, C. E. & Suhr, J. A.
    (1994) Social support communication in the context of marriage: An analysis of couples’ supportive interactions. InB. R. Burleson, T. L. Albrecht, & I. G. Sarason (Eds.), Communication of social support: Messages, interactions, relationships, and community (pp.113–135). Thousand Oaks, CA: Sage.
     [Google Scholar]
  22. DeCapua, A., & Dunham, J. F.
    (2007) The pragmatics of advice giving: Cross-cultural perspectives. Intercultural Pragmatics, 4(3), 319–342. 10.1515/IP.2007.016
     https://doi.org/10.1515/IP.2007.016 [Google Scholar]
  23. Eslami-Rasekh, Z.
    (2005) Raising the pragmatic awareness of language learners. ELT Journal, 59(3), 199–208. 10.1093/elt/cci039
     https://doi.org/10.1093/elt/cci039 [Google Scholar]
  24. Flöck, I., & Geluykens, R.
    (2015) Speech acts in corpus pragmatics: A quantitative contrastive study of directives in spontaneous and elicited discourse. InJ. Romero-Trillo (Ed.), Yearbook of corpus linguistics and pragmatics 2015: Current approaches to discourse and translation studies (pp.7–37). Cham: Springer. 10.1007/978‑3‑319‑17948‑3_2
     https://doi.org/10.1007/978-3-319-17948-3_2 [Google Scholar]
  25. Golato, A.
    (2003) Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied Linguistics, 24(1), 90–121. 10.1093/applin/24.1.90
     https://doi.org/10.1093/applin/24.1.90 [Google Scholar]
  26. Goldsmith, D. J.
    (2004) Communicating social support. New York, NY: Cambridge University Press. 10.1017/CBO9780511606984
     https://doi.org/10.1017/CBO9780511606984 [Google Scholar]
  27. Grabowski, K. C.
    (2007) Reconsidering the measurement of pragmatic knowledge using a reciprocal written test format. Columbia University Working Papers in TESOL & Applied Linguistics, 7(1), 1–48.
     [Google Scholar]
  28. Halenko, N., & Jones, C.
    (2011) Teaching pragmatic awareness of spoken requests to Chinese EAP learners in the UK: Is explicit instruction effective?System, 39(2), 240–250. 10.1016/j.system.2011.05.003
     https://doi.org/10.1016/j.system.2011.05.003 [Google Scholar]
  29. Hartford, B. S., & Bardovi-Harlig, K.
    (1992) Experimental and observational data in the study of interlanguage pragmatics. Pragmatics and Language Learning, 3, 33–52.
     [Google Scholar]
  30. Hinkel, E.
    (1997) Appropriateness of advice: DCT and multiple choice data. Applied Linguistics, 18(1), 1–26. 10.1093/applin/18.1.1
     https://doi.org/10.1093/applin/18.1.1 [Google Scholar]
  31. Hong, C. Y., & Shih, S. C.
    (2013) Proficiency and complaints: Analyses of production and perceptions. Intergrams, 14(1), 1–20.
     [Google Scholar]
  32. Johnston, B., Kasper, G., & Ross, S.
    (1998) Effect of rejoinders in production questionnaires. Applied Linguistics, 19(2), 157–182. 10.1093/applin/19.2.157
     https://doi.org/10.1093/applin/19.2.157 [Google Scholar]
  33. Kasper, G., & Dahl, M.
    (1991) Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 13(2), 215–247. 10.1017/S0272263100009955
     https://doi.org/10.1017/S0272263100009955 [Google Scholar]
  34. Kouper, I.
    (2010) The pragmatics of peer advice in a Live Journal community. Language@ internet, 7, article 1.
     [Google Scholar]
  35. Labben, A.
    (2016) Reconsidering the development of the discourse completion test in interlanguage pragmatics. Pragmatics: Quarterly Publication of the International Pragmatics Association (IPrA), 26(1), 69–91. 10.1075/prag.26.1.04lab
     https://doi.org/10.1075/prag.26.1.04lab [Google Scholar]
  36. Martínez-Flor, A. M.
    (2003) Non-native speakers’ production of advice acts: The effects of proficiency. Revista Española de Lingüística Aplicada, 16, 139–153.
     [Google Scholar]
  37. McHugh, M. L.
    (2013) The chi-square test of independence. Biochemia Medica, 23(2), 143–149. 10.11613/BM.2013.018
     https://doi.org/10.11613/BM.2013.018 [Google Scholar]
  38. Nguyen, T. T. M.
    (2019) Data collection methods in L2 pragmatics research: An overview. InN. Taguchi (Ed.), The Routledge handbook of SLA and pragmatics (pp.195–211). New York, NY: Routledge.
     [Google Scholar]
  39. Parvaresh, V., & Tavakoli, M.
    (2009) Discourse completion tasks as elicitation tools: How convergent are they. The Social Sciences, 4(4), 366–373.
     [Google Scholar]
  40. Patton, M. Q.
    (1990) Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage.
     [Google Scholar]
  41. Plonsky, L., & Oswald, F. L.
    (2014) How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. 10.1111/lang.12079
     https://doi.org/10.1111/lang.12079 [Google Scholar]
  42. R Core Team
    R Core Team (2013) R: A language and environment for statistical computing (Version 3.4.3) [Computer software]. Vienna, Austria: R Foundation for Statistical Computing.
     [Google Scholar]
  43. Schauer, G. A., & Adolphs, S.
    (2006) Expressions of gratitude in corpus and DCT data: Vocabulary, formulaic sequences, and pedagogy. System, 34(1), 119-134.
     [Google Scholar]
  44. Staples, S., & Fernández, J.
    (2018) Corpus linguistics approaches to L2 pragmatics. InN. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (241–254). Routledge: London.
     [Google Scholar]
  45. Woodfield, H.
    (2008) Problematising discourse completion tasks: Voices from verbal report. Evaluation & Research in Education, 21(1), 43–69. 10.2167/eri413.0
     https://doi.org/10.2167/eri413.0 [Google Scholar]
  46. Yuan, Y.
    (2001) An inquiry into empirical pragmatics data-gathering methods: Written DCTs, oral DCTs, field notes, and natural conversations. Journal of Pragmatics, 33(2), 271–292. 10.1016/S0378‑2166(00)00031‑X
     https://doi.org/10.1016/S0378-2166(00)00031-X [Google Scholar]
  47. (2002) Compliments and compliment responses in Kunming Chinese. Pragmatics: Quarterly Publication of the International Pragmatics Association (IPrA), 12(2), 183–226. 10.1075/prag.12.2.04yua
     https://doi.org/10.1075/prag.12.2.04yua [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/ap.19009.has
Loading
Using a corpus in creating and evaluating a DCT
Applied Pragmatics 2, 80 (2020); https://doi.org/10.1075/ap.19009.has
/content/journals/10.1075/ap.19009.has
/content/journals/10.1075/ap.19009.has
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): advice-giving; authenticity; corpus-based instrument; discourse completion task; pragmatics; technology-mediated communication

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error