1887
Volume 26, Issue 1
  • ISSN 1018-2101
  • E-ISSN: 2406-4238

Abstract

A survey of the field of Interlanguage Pragmatics (ILP) shows that the Discourse Completion Test (DCT)2, also referred to as a ‘discourse completion task’ or a ‘production questionnaire’, has been the most frequently used instrument to evaluate second/foreign language learners’ ability to perform speech acts in a target language, despite the harsh criticism leveled against its low construct validity and its failure to represent the features of authentic discourse. Interestingly, focusing on the statement of objectives of a number of ILP studies using DCTs, one can notice that such studies rarely refer to the DCT as a language test. In addition, an overview of the DCT design process as described in several ILP studies shows that ever since its adaptation for the study of pragmatic ability (Blum-Kulka, 1982), there has been a tendency to use or adapt one of the existing DCT versions used in previous studies based on the argument of comparability of results. While a number of ILP researchers tried to improve the design of the DCT by the inclusion of rejoinders or by enhancing the prompt material (e.g. Billmyer and Varghese, 2000), few attempts have been made to reconsider the DCT development process. McNamara and Roever (2006: 253) urge for the need for “more research on testing of sociopragmatic knowledge and design of discourse completion tests for testing purposes.”The present paper starts with an overview of the literature about DCTs with special reference to the cognitive validity of the instrument and to previous studies dealing with DCT structure and content. Then, with reference to research in the fields of language testing and psychometrics, it shows that, whether used for research or instructional purposes, the DCT shares several qualities with language tests. As such, it is argued that the DCT should be treated as a language test and not as a questionnaire and should, thus, undergo a rigorous developmental process. Based on recent models of language test construction, the paper concludes with an overview of the stages of DCT development.

Loading

Article metrics loading...

/content/journals/10.1075/prag.26.1.04lab
2016-03-01
2019-10-24
Loading full text...

Full text loading...

References

  1. Bachman, L.F. , and A.S. Palmer
    (1996) Language testing in practice. Oxford: Oxford University Press.
    [Google Scholar]
  2. Bachman, L.F
    (2004) Statistical analyses for language assessment. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511667350
    https://doi.org/10.1017/CBO9780511667350 [Google Scholar]
  3. Bardovi-Harlig, K. , and B.S. Hartford
    (1993) Learning the rules of academic talk: A longitudinal study of pragmatic change. Studies in Second Language Acquisition15.3: 279-304. doi: 10.1017/S0272263100012122
    https://doi.org/10.1017/S0272263100012122 [Google Scholar]
  4. Bardovi‐Harlig, K
    (2013) Developing L2 pragmatics. Language Learning63.1: 68-86. doi: 10.1111/j.1467‑9922.2012.00738.x
    https://doi.org/10.1111/j.1467-9922.2012.00738.x [Google Scholar]
  5. Bax, S
    (2013) Readers’ cognitive processes during IELTS reading tests: Evidence from eye tracking. ELT Research Papers13-06.
    [Google Scholar]
  6. Beebe, L.M. , and M.C. Cummings
    (1985) Speech act performance: A function of the data collection procedure?Paper presented at the TESOL convention , New York.
    [Google Scholar]
  7. Bella, S
    (2014) A contrastive study of apologies performed by Greek native speakers and English learners of Greek as a foreign language. Pragmatics24.1: 679-713. doi: 10.1075/prag.24.4.01bel
    https://doi.org/10.1075/prag.24.4.01bel [Google Scholar]
  8. Bergman, M.L. , and G. Kasper
    (1993) Perception and performance in native and non-native apology. In G. Kasper , and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp.82-107.
    [Google Scholar]
  9. Bialystok, E
    (1993) Symbolic representation and attentional control. In G. Kaspe,r and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp.43-57.
    [Google Scholar]
  10. Billmyer, K. , and M. Varghese
    (2000) Investigating instrument-based pragmatic variability: Effects of enhancing discourse completion tests. Applied Linguistics21.4: 517-552. doi: 10.1093/applin/21.4.517
    https://doi.org/10.1093/applin/21.4.517 [Google Scholar]
  11. Blum-Kulka, S
    (1982) Learning to say what you mean in a second language: A study of the speech act performance of learners of Hebrew as a second language. Applied Linguistics3.1: 29-59. doi: 10.1093/applin/3.1.29
    https://doi.org/10.1093/applin/3.1.29 [Google Scholar]
  12. Blum-Kulka, S. , J. House , and G. Kasper
    (eds.) (1989) Cross-cultural pragmatics: Requests and apologies. Norwood, NJ: Albex.
    [Google Scholar]
  13. Boxer, D
    (2002) Discourse issues in cross-cultural pragmatics. Annual Review of Applied Linguistics22: 150-167. doi: 10.1017/S0267190502000089
    https://doi.org/10.1017/S0267190502000089 [Google Scholar]
  14. Brown, J.D
    (2008) Raters, functions, item types and the dependability of L2 pragmatics tests. In E.A. Soler , and A.M. Flor (eds.), Investigating pragmatics in foreign language learning, teaching and testing (Vol. 30). Multilingual Matters. 224-48.
    [Google Scholar]
  15. Callies, M
    (2013) Advancing the research agenda of Interlanguage Pragmatics: The role of learner corpora. InYearbook of Corpus Linguistics and Pragmatics 2013. New York: Springer, pp.9-36. doi: 10.1007/978‑94‑007‑6250‑3_2
    https://doi.org/10.1007/978-94-007-6250-3_2 [Google Scholar]
  16. Cohen, A.D
    (1996) Developing the ability to perform speech acts. Studies in Second Language Acquisition18.2: 253-269. doi: 10.1017/S027226310001490X
    https://doi.org/10.1017/S027226310001490X [Google Scholar]
  17. (1996) Speech acts. In N. Hornberger , and S. McKay (eds.), Sociolinguistics and language teaching. Cambridge: Cambridge University Press, pp.382-419.
    [Google Scholar]
  18. (2004) Assessing speech acts in a second language. In D. Boxer , and A.D. Cohen (eds.), Studying speaking to inform second language learning (Vol. 8). Multilingual Matters302-327.
    [Google Scholar]
  19. (2008) Teaching and assessing L2 pragmatics: What can we expect from learners?Language Teaching41.2: 213-235. doi: 10.1017/S0261444807004880
    https://doi.org/10.1017/S0261444807004880 [Google Scholar]
  20. Cohen, A.D. , and E. Olshtain
    (1994) Researching the production of second-language speech acts. Research methodology in second-language acquisition143-156.
    [Google Scholar]
  21. Dornyei, Z
    (2003) Questionnaires in second Language Research: Construction, administration, and processing. Mahwah, NJ: Lawrence Erlbraum.
    [Google Scholar]
  22. Douglas, D
    (2000) Assessing languages for specific purposes. Cambridge, UK: Cambridge University Press.
    [Google Scholar]
  23. Ellis, R
    (2004) The definition and measurement of L2 explicit knowledge. Language Learning54 : 227–275. doi: 10.1111/j.1467‑9922.2004.00255.x
    https://doi.org/10.1111/j.1467-9922.2004.00255.x [Google Scholar]
  24. Faerch. C. , and G. Kasper
    (1984) Pragmatic knowledge: Rules and procedures. Applied Linguistics5.3: 214-225. doi: 10.1093/applin/5.3.214
    https://doi.org/10.1093/applin/5.3.214 [Google Scholar]
  25. Foster, P. , and P. Skehan
    (1996) The influence of planning and task type on second language performance. Studies in Second language acquisition18.3: 299-323. doi: 10.1017/S0272263100015047
    https://doi.org/10.1017/S0272263100015047 [Google Scholar]
  26. Golato, A
    (2003) Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied linguistics24.1: 90-121. doi: 10.1093/applin/24.1.90
    https://doi.org/10.1093/applin/24.1.90 [Google Scholar]
  27. Grabowski, K.C
    (2007) Reconsidering the measurement of pragmatic knowledge using a reciprocal written task format. Columbia University Working Papers in TESOL & Applied Linguistics7.1: 1-48.
    [Google Scholar]
  28. (2008) Investigating the construct validity of a performance test designed to measure grammatical and pragmatic knowledge. Spaan Fellow Working Papers in Foreign Language Assessment6: 131-179.
    [Google Scholar]
  29. Hinkel, E
    (1997) Appropriateness of advice: DCT and multiple choice data. Applied linguistics18.1: 1-26. doi: 10.1093/applin/18.1.1
    https://doi.org/10.1093/applin/18.1.1 [Google Scholar]
  30. Hendriks, B
    (2008) Dutch English requests: A study of request performance by Dutch learners of English. In M. Pütz , and J. Neff-van Aertselaer (eds.), Developing contrastive pragmatics: Interlanguage and cross-cultural perspectives (Vol. 31). Berlin: Walter de Gruyter, pp.335-354.
    [Google Scholar]
  31. Hudson, T. , E. Detmer , and J.D. Brown
    (1995) Developing prototypic measures of crosscultural pragmatics (Tech. Rep. No 7). Honolulu: University of Hawai’i, Second Language Teaching and Curriculum Center.
    [Google Scholar]
  32. Hymes, D
    (1972) On communicative competence. Sociolinguistics269-293.
    [Google Scholar]
  33. Johnston, B. , G. Kasper ,and S. Ross
    (1998) Effect of rejoinders in production questionnaires. Applied Linguistics19.2: 157-182. doi: 10.1093/applin/19.2.157
    https://doi.org/10.1093/applin/19.2.157 [Google Scholar]
  34. Kasper, G
    (1997) The role of pragmatics in language teacher education. In K. Bardovi-Harlig , and B. Hartford (eds.), Beyond methods. Components of second language teacher education. New York: McGraw Hill, pp.113-141.
    [Google Scholar]
  35. (2010) Interlanguage pragmatics. In M. Fried , J.O. Östman , and J. Verschueren (eds.), Variation and change: Pragmatic perspectives (Vol. 6). Amsterdam: John Benjamins Publishing Company, pp.141-154. doi: 10.1075/hoph.6.11kas
    https://doi.org/10.1075/hoph.6.11kas [Google Scholar]
  36. Kasper, G. , and M. Dahl
    (1991) Research methods in interlanguage pragmatics. Studies in Second Language Acquisition13: 215-247. doi: 10.1017/S0272263100009955
    https://doi.org/10.1017/S0272263100009955 [Google Scholar]
  37. Kasper, G. , and S. Blum-Kulka
    (eds.) (1993) Interlanguage pragmatics. Oxford: Oxford University Press.
    [Google Scholar]
  38. Kasper, G. , and S. Ross
    (2013) Assessing second language pragmatics: An overview and introductions. In S. Ross , and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan, pp.1-40. doi: 10.1057/9781137003522_1
    https://doi.org/10.1057/9781137003522_1 [Google Scholar]
  39. Korsko, P
    (2004) The narrative shape of two-party complaints in Portuguese: A discourse analytic study. Unpublished doctoral dissertation, Teachers College, Columbia University, New York City.
    [Google Scholar]
  40. Leech, G
    (1983) Principles of pragmatics. London: Longman.
    [Google Scholar]
  41. Lin, M.F
    (2014) An interlanguage pragmatic study on Chinese EFL learners’ refusal: Perception and performance. Journal of Language Teaching and Research5.3: 642-653. doi: 10.4304/jltr.5.3.642‑653
    https://doi.org/10.4304/jltr.5.3.642-653 [Google Scholar]
  42. Martínez-Flor, A. , and E. Usó-Juan
    (2006) Learners’ use of request modifiers across two University ESP disciplines. Ibérica12: 23-41.
    [Google Scholar]
  43. McNamara, T.F. , and C. Roever
    (2006) Language testing: The social dimension. Oxford: Blackwell.
    [Google Scholar]
  44. Olshtain, E. , and L. Weinbach
    (1993) Interlanguage features of the speech act of complaining. In G. Kasper , and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press.
    [Google Scholar]
  45. Rintell, E. , and C.L. Mitchell
    (1989) Studying requests and apologies: An inquiry into methods. In S. Blum-Kulka. , J. House , and G. Kasper (eds.), Cross-cultural pragmatics: Requests and apologies. Norwood, NJ: Albex, pp.248-72.
    [Google Scholar]
  46. Roever, C
    (2004) Difficulty and practicality in tests of interlanguage pragmatics. In D. Boxer , and A.D. Cohen . (eds.), Studying speaking to inform second language learning (Vol. 8). Multilingual Matters283-301.
    [Google Scholar]
  47. (2006) Validation of a web-based test of ESL pragmalinguistics. Language Testing23.2: 229-256. doi: 10.1191/0265532206lt329oa
    https://doi.org/10.1191/0265532206lt329oa [Google Scholar]
  48. (2007) DIF in the assessment of second language pragmatics. Language Assessment Quarterly4.2: 165-189. doi: 10.1080/15434300701375733
    https://doi.org/10.1080/15434300701375733 [Google Scholar]
  49. (2008) Rater, item and candidate effects in discourse completion tests: A FACETS approach. In E.A. Soler , and A.M. Flor (eds.), Investigating pragmatics in foreign language learning, teaching and testing (Vol. 30). Multilingual Matters249-266.
    [Google Scholar]
  50. (2010) Effects of cultural background in a test of ESL pragmalinguistics: A DIF approach. Pragmatics and language learning12.
    [Google Scholar]
  51. (2011) Testing of second language pragmatics: Past and future. Language Testing28.4: 463-481. doi: 10.1177/0265532210394633
    https://doi.org/10.1177/0265532210394633 [Google Scholar]
  52. Rose, K.R
    (1992) Speech acts and questionnaires: The effect of hearer response. Journal of pragmatics17.1: 49-62. doi: 10.1016/0378‑2166(92)90028‑A
    https://doi.org/10.1016/0378-2166(92)90028-A [Google Scholar]
  53. (1994) On the validity of discourse completion tests in non-Western contexts. Applied Linguistics15.1: 1-14. doi: 10.1093/applin/15.1.1
    https://doi.org/10.1093/applin/15.1.1 [Google Scholar]
  54. Rose, K.R. , and R. Ono
    (1995) Eliciting speech act data in Japanese: The effect of questionnaire type. Language learning45.2: 191-223. doi: 10.1111/j.1467‑1770.1995.tb00438.x
    https://doi.org/10.1111/j.1467-1770.1995.tb00438.x [Google Scholar]
  55. Ross, S. , and G. Kasper
    (eds.) (2013) Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan. doi: 10.1057/9781137003522
    https://doi.org/10.1057/9781137003522 [Google Scholar]
  56. Rylander, J. , P. Clark , and R. Derrah
    (2013) A video-based method of assessing pragmatic awareness. In S. Ross , and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan, pp.65-97. doi: 10.1057/9781137003522_3
    https://doi.org/10.1057/9781137003522_3 [Google Scholar]
  57. Salehi, R
    (2014) A comparative analysis of apology strategy: Iranian EFL learners and native English speakers. Procedia-Social and Behavioral Sciences98: 1658-1665. doi: 10.1016/j.sbspro.2014.03.590
    https://doi.org/10.1016/j.sbspro.2014.03.590 [Google Scholar]
  58. Samavarchi, L. , and H. Allami
    (2012) Giving condolences by Persian EFL learners: A contrastive sociopragmatic study. International Journal of English Linguistics2.1: 71-78. doi: 10.5539/ijel.v2n1p71
    https://doi.org/10.5539/ijel.v2n1p71 [Google Scholar]
  59. Sasaki, M
    (1998) Investigating EFL students’ production of speech acts: A comparison of production questionnaires and role plays. Journal of Pragmatics30.4: 457-484. doi: 10.1016/S0378‑2166(98)00013‑7
    https://doi.org/10.1016/S0378-2166(98)00013-7 [Google Scholar]
  60. Schmidt, R
    (1993) Consciousness, learning, and interlanguage pragmatics. In G. Kasper , and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp.21-42.
    [Google Scholar]
  61. Thomas, J
    (1983) Cross-cultural pragmatic failure. Applied Linguistics4: 91-112. doi: 10.1093/applin/4.2.91
    https://doi.org/10.1093/applin/4.2.91 [Google Scholar]
  62. Walters, F.S
    (2013) Interfaces between a discourse completion test and a conversation analysis-informed test of L2 pragmatic competence. In S. Ross , and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan.
    [Google Scholar]
  63. Wolfson, N
    (1989) Perspectives: Sociolinguistics and TESOL. New York: Newsbury House.
    [Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.1075/prag.26.1.04lab
Loading
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error