1887
  1. Home
  2. e-Journals
  3. Applied Pragmatics
  4. Online First Articles
  5. Online First Article
image of Exploring pragmatic competence in “New Italians”
PDF
EPUB

Abstract

This study examines the pragmatic competence of L2 speakers of Italian ( = 101) who successfully passed the Certit B1 exam, focusing on their oral production of refusals and cancellations in voice messages. The data, collected from authentic exam performances, were analyzed through two complementary approaches: an analysis based on pragmatic categories identified in previous work and an evaluation by four non-expert L1 raters. The messages were annotated using a mixed human–LLM procedure, allowing for fine-grained coding of pragmatic categories. Results show that L2 speakers tend to produce long and elaborated messages, often characterized by redundancy and extensive use of greetings and address terms. While raters tolerated diverse realization strategies, they penalized incoherence and ambiguity, particularly in terms of unclear illocutionary force. The findings highlight the need for assessment models sensitive to multilingual pragmatics and for pedagogical practices that promote coherence and efficiency in L2 speech act performance.

© John Benjamins Publishing Company
Available under the CC BY 4.0 license.
Loading

Article metrics loading...

/content/journals/10.1075/ap.26004.bro
2026-07-03
2026-07-21

  • From This Site

    /content/journals/10.1075/ap.26004.bro
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

/deliver/fulltext/10.1075/ap.26004.bro/ap.26004.bro.html?itemId=/content/journals/10.1075/ap.26004.bro&mimeType=html&fmt=ahah

References

  1. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice: Designing and developing useful language tests. Oxford University Press.
     [Google Scholar]
  2. Bardovi-Harlig, K., & Dörnyei, Z.
    (1998) Do language learners recognize pragmatic violations? Pragmatic vs. grammatical awareness in instructed L2 learning. TESOL Quarterly, (), –. 10.2307/3587583
     https://doi.org/10.2307/3587583 [Google Scholar]
  3. Bardovi-Harlig, K., & Hartford, B. S.
    (1991) Saying “no” in English: Native and nonnative rejections. Pragmatics and Language Learning: Monograph Series, , –.
     [Google Scholar]
  4. Beebe, L. M., Takahashi, T., & Uliss-Weltz, R.
    (1990) Pragmatic transfer in ESL refusals. InR. C. Scarcella, E. Anderson, & S. D. Krashen (Eds.), Developing communicative competence in a second language (pp.–). Newbury House.
     [Google Scholar]
  5. Bella, S.
    (2011) Mitigation and politeness in Greek invitation refusals: Effects of length of residence in the target community and intensity of interaction on non-native speakers’ performance. Journal of Pragmatics, (), –. 10.1016/j.pragma.2010.11.005
     https://doi.org/10.1016/j.pragma.2010.11.005 [Google Scholar]
  6. (2012) Pragmatic development in a foreign language: A study of Greek FL refusals. Journal of Pragmatics, (), –. 10.1016/j.pragma.2012.08.014
     https://doi.org/10.1016/j.pragma.2012.08.014 [Google Scholar]
  7. (2014) Developing the ability to refuse: A cross-sectional study of Greek FL refusals. Journal of Pragmatics, , –. 10.1016/j.pragma.2013.11.015
     https://doi.org/10.1016/j.pragma.2013.11.015 [Google Scholar]
  8. Bianco, A., Brocca, N., & Garassino, D.
    (2026) Assessing the potential of LLM-assisted annotation for corpus pragmatics: The case of humor. Corpus Pragmatics, , Article 33. 10.1007/s41701‑026‑00235‑7
     https://doi.org/10.1007/s41701-026-00235-7 [Google Scholar]
  9. Bley-Vroman, R.
    (1983) The comparative fallacy in interlanguage studies: The case of systematicity. Language Learning, (), –. 10.1111/j.1467‑1770.1983.tb00983.x
     https://doi.org/10.1111/j.1467-1770.1983.tb00983.x [Google Scholar]
  10. Brocca, N.
    (2021) LADDER: Un corpus di scritture digitali per l’insegnamento della pragmatica in L2. Un esempio di analisi di disdette in WhatsApp. Italiano LinguaDue, (), –.
     [Google Scholar]
  11. (2024) LADDER: A corpus for pragmatic competences in Italian L1/L2 [Data set]. https://hdl.handle.net/21.11115/0000-0011-BF1F-5
     [Google Scholar]
  12. Brocca, N., Nuzzo, E., Cortés Velásquez, D., & Rudigier, M.
    (2023) Linguistic politeness across Austria and Italy: Backing out of an invitation with an instant message. Journal of Pragmatics, , –. 10.1016/j.pragma.2023.02.018
     https://doi.org/10.1016/j.pragma.2023.02.018 [Google Scholar]
  13. Brocca, N., Nuzzo, E., & Wang, J.
    (2026) AI-driven speech act annotation: Accuracy and reproducibility across ChatGPT, LadderWeb and LLaMA. AI-Linguistica. Linguistic Studies on AI-Generated Texts and Discourses, (). 10.62408/ai‑ling.v3i1.33
     https://doi.org/10.62408/ai-ling.v3i1.33 [Google Scholar]
  14. (under review). Machine learning for pragmatic annotation: Comparing supervised and pre-trained models in speech act tagging Lodz Papers in Pragmatics.
     [Google Scholar]
  15. Brown, P., & Levinson, S. C.
    (1987) Politeness: Some universals in language usage. Cambridge University Press. 10.1017/CBO9780511813085
     https://doi.org/10.1017/CBO9780511813085 [Google Scholar]
  16. Canale, M., & Swain, M.
    (1980) Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, , –. 10.1093/applin/1.1.1
     https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  17. Chen, X., Lei, Y., & Zhang, Z.
    (1995) Refusing in Chinese. InG. Kasper (Ed.), Pragmatics of Chinese as native and target language (pp.–). University of Hawaii Press.
     [Google Scholar]
  18. Cognigni, E.
    (2014) Migrazione femminile e bisogni di apprendimento in italiano L2 Uno studio di caso presso le donne del ricongiungimento familiare. EL.LE. Educazione Linguistica. Language Education, (), –.
     [Google Scholar]
  19. Council of Europe
    Council of Europe (2020) Common European Framework of Reference for Languages: Companion volume. Council of Europe. https://www.coe.int/en/web/common-european-framework-reference-languages/cefr-companion-volume-and-its-language-versions
     [Google Scholar]
  20. Cook, V.
    (1999) Going beyond the native speaker in language teaching. TESOL Quarterly, (), –. 10.2307/3587717
     https://doi.org/10.2307/3587717 [Google Scholar]
  21. Cortés Velásquez, D., & Nuzzo, E.
    (2022) Declining an invitation: The pragmatics of Italian and Colombian Spanish. InS. Gesuato, G. Salvato, & E. Castello (Eds.), Pragmatic aspects of L2 communication: From awareness through description to assessment (pp.–). Cambridge Scholars Publishing.
     [Google Scholar]
  22. (2024) Disdir: Cancellations and other refusal strategies [Data set]. ARCHE. https://hdl.handle.net/21.11115/0000-0011-BF16-E
     [Google Scholar]
  23. European Union
    European Union (2016) Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). Official Journal of the European Union, L, –. https://eur-lex.europa.eu/eli/reg/2016/679/oj
     [Google Scholar]
  24. Fang, X.
    (2024) Making refusals via English as a lingua franca: Chinese English speakers’ strategies and sequences. Journal of Pragmatics, , –. 10.1016/j.pragma.2024.06.011
     https://doi.org/10.1016/j.pragma.2024.06.011 [Google Scholar]
  25. Félix-Brasdefer, J. C.
    (2003) Declining an invitation: A cross-cultural study of pragmatic strategies in American English and Latin American Spanish. Multilingua, (), –. 10.1515/mult.2003.012
     https://doi.org/10.1515/mult.2003.012 [Google Scholar]
  26. (2006) Linguistic politeness in Mexico: Refusal strategies among male speakers of Mexican Spanish. Journal of Pragmatics, (), –. 10.1016/j.pragma.2006.05.004
     https://doi.org/10.1016/j.pragma.2006.05.004 [Google Scholar]
  27. (2008) Sociopragmatic variation: Dispreferred responses in Mexican and Dominican Spanish. Journal of Politeness Research, (), –. 10.1515/PR.2008.004
     https://doi.org/10.1515/PR.2008.004 [Google Scholar]
  28. (2013) Refusing in L2 Spanish: The effects of the context of learning during a short-term study abroad program. InO. Martí-Arnándiz & P. Salazar-Campillo (Eds.), Refusals in instructional contexts and beyond (Vol., pp.–). Utrecht Studies in Language and Communication. 10.1163/9789401209717_009
     https://doi.org/10.1163/9789401209717_009 [Google Scholar]
  29. Frescura, M.
    (1997) Strategie di rifiuto in italiano: Uno studio etnografico. Italica, (), –. 10.2307/479483
     https://doi.org/10.2307/479483 [Google Scholar]
  30. Gass, S., & Houck, N.
    (1999) Interlanguage refusals: A cross-cultural study of Japanese-English. Mouton de Gruyter.
     [Google Scholar]
  31. Grabowski, K. C.
    (2016) Assessing pragmatic competence. InD. Tsagari & J. Banerjee (Eds.), Handbook of second language assessment (pp.–). De Gruyter Mouton. 10.1515/9781614513827‑013
     https://doi.org/10.1515/9781614513827-013 [Google Scholar]
  32. Grice, H. P.
    (1975) Logic and conversation. InP. Cole & J. L. Morgan (Eds.), Syntax and semantics 3: Speech acts (pp.–). Academic Press. 10.1163/9789004368811_003
     https://doi.org/10.1163/9789004368811_003 [Google Scholar]
  33. House, J.
    (1996) Developing pragmatic fluency in English as a foreign language: Routines and metapragmatic awareness. Studies in Second Language Acquisition, (), –. 10.1017/S0272263100014893
     https://doi.org/10.1017/S0272263100014893 [Google Scholar]
  34. Kasper, G., & Roever, C.
    (2005) Pragmatics in second language learning. InE. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp.–). Routledge.
     [Google Scholar]
  35. Koo, T. K., & Li, M. Y.
    (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, (), –. 10.1016/j.jcm.2016.02.012
     https://doi.org/10.1016/j.jcm.2016.02.012 [Google Scholar]
  36. Landis, J. R., & Koch, G. G.
    (1977) The measurement of observer agreement for categorical data. Biometrics, (), –. 10.2307/2529310
     https://doi.org/10.2307/2529310 [Google Scholar]
  37. Nelson, G. L., Carson, J., Al Batal, M., & El Bakary, W.
    (2002) Cross-cultural pragmatics: Strategy use in Egyptian Arabic and American English refusals. Applied Linguistics, (), –. 10.1093/applin/23.2.163
     https://doi.org/10.1093/applin/23.2.163 [Google Scholar]
  38. Nuzzo, E., Brocca, N., & Cortés Velásquez, D.
    (2025) Refusing and canceling an appointment in Italian as a first language: What models for the assessment of learners’ performances?Italiano Lingua Due, (), –. 10.54103/2037‑3597/30518
     https://doi.org/10.54103/2037-3597/30518 [Google Scholar]
  39. Nuzzo, E., & Cortés Velásquez, D.
    (2020) Canceling last minute in Italian and Colombian Spanish: A cross-cultural account of pragmalinguistic strategies. Corpus Pragmatics, , –. 10.1007/s41701‑020‑00084‑y
     https://doi.org/10.1007/s41701-020-00084-y [Google Scholar]
  40. Nuzzo, E., Cortés Velásquez, D., & Brocca, N.
    (under review). Assessing pragmatic effectiveness in learner Italian: Native speaker judgments of refusals and cancellations in a B1 exam corpus. e-JournALL.
     [Google Scholar]
  41. Purpura, J. E.
    (2004) Assessing grammar. Cambridge University Press. 10.1017/CBO9780511733086
     https://doi.org/10.1017/CBO9780511733086 [Google Scholar]
  42. Roever, C.
    (2011) Testing of second language pragmatics: Past and future. Language Testing, (), –. 10.1177/0265532210394633
     https://doi.org/10.1177/0265532210394633 [Google Scholar]
  43. Searle, J. R.
    (1977) Speech acts: An essay in the philosophy of language. Cambridge University Press.
     [Google Scholar]
  44. Sickinger, P., & Renkwitz, K.
    (2021) How good are native speakers at evaluating pragmatic competence? Native speaker assessments of learner status and appropriate pragmatic behaviour English. Langages, (), –. 10.3917/lang.222.0113
     https://doi.org/10.3917/lang.222.0113 [Google Scholar]
  45. Su, Y., & Chen, X.
    (2024) Examining refusal and acceptance sequences employing a data-driven binary rating approach. Research Methods in Applied Linguistics, (), Article 100161. 10.1016/j.rmal.2024.100161
     https://doi.org/10.1016/j.rmal.2024.100161 [Google Scholar]
  46. Taguchi, N.
    (2011a) Rater variation in the assessment of speech acts. Pragmatics, (), –. 10.1075/prag.21.3.08tag
     https://doi.org/10.1075/prag.21.3.08tag [Google Scholar]
  47. (2011b) The effect of L2 proficiency and study-abroad experience on pragmatic comprehension. Language Learning, (), –. 10.1111/j.1467‑9922.2011.00633.x
     https://doi.org/10.1111/j.1467-9922.2011.00633.x [Google Scholar]
  48. Taguchi, N., & Caprario, M.
    (2023) L2 pragmatics research and the problem of L1 norms. InC. A. Chapelle (Ed.), The encyclopedia of applied linguistics. John Wiley & Sons. 10.1002/9781405198431.wbeal20047
     https://doi.org/10.1002/9781405198431.wbeal20047 [Google Scholar]
  49. Taguchi, N., & Li, S.
    (2020) Contrastive pragmatics and second language (L2) pragmatics: Approaches to assessing L2 speech act production. Contrastive Pragmatics, (), –. 10.1163/26660393‑BJA10014
     https://doi.org/10.1163/26660393-BJA10014 [Google Scholar]
  50. Verzella, M., & Tommaso, L.
    (2020) The pragmatics of refusing a request in Italian and American English: A comparative study. Discourse and Interaction, (), –. 10.5817/DI2020‑1‑92
     https://doi.org/10.5817/DI2020-1-92 [Google Scholar]
  51. Yu, D., Li, L., Su, H., & Fuoli, M.
    (2024) Assessing the potential of LLM-assisted annotation for corpus-based pragmatics and discourse analysis: The case of apology. International Journal of Corpus Linguistics, (), –. 10.1075/ijcl.23087.yu
     https://doi.org/10.1075/ijcl.23087.yu [Google Scholar]
/content/journals/10.1075/ap.26004.bro
Loading
Exploring pragmatic competence in “New Italians”
John Benjamins ; https://doi.org/10.1075/ap.26004.bro
/content/journals/10.1075/ap.26004.bro
/content/journals/10.1075/ap.26004.bro
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: refusals ; pragmatic competence ; language assessment ; Italian as a second language ; cancellations

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error