Volume 3, Issue 1
  • ISSN 2215-1478
  • E-ISSN: 2215-1486
Buy:$35.00 + Taxes


This study aims to provide the first empirical assessment of quantitative research methods and study quality in learner corpus research. We systematically review quantitative primary studies referenced in the (LCB), a representative bibliography of learner corpus research maintained by the which contained 1,276 references when the current study began. Each primary study in the LCB was coded for over fifty features representing six dimensions: (a) publication type (i.e. conference paper, book chapter, journal article), (b) research focus (e.g. lexis, grammar), (c) methodological features (e.g. keyword analysis, error analysis, use of reference corpus), (d) statistical analyses (e.g. X², -test, regression analysis), and (e) reporting practices (e.g. reliability coefficients, means). Results point to several systematic strengths as well as many flaws, such as the absence of research questions, incomplete and inconsistent reporting practices (e.g. means without standard deviations), and lack of statistical literacy (i.e. LCR studies generally overrely on tests of statistical significance, do not report effect sizes, rarely check or report whether statistical assumptions have been met, and rarely use multivariate analyses). Improvements over time, however, are clearly noted and there are signs that, like other related disciplines, learner corpus research is slowly undergoing methodological reform.


Article metrics loading...

Loading full text...

Full text loading...


  1. APA Publications and Communications Board Working Group on Journal Article Reporting Standards
    APA Publications and Communications Board Working Group on Journal Article Reporting Standards 2008 “Reporting standards for research in psychology”, American Psychologist63(9), 839–851. doi: 10.1037/0003‑066X.63.9.839
    https://doi.org/10.1037/0003-066X.63.9.839 [Google Scholar]
  2. Baayen, R. H.
    2001Word Frequency Distributions. Dordrecht: Kluwer Academic Publishers. doi: 10.1007/978‑94‑010‑0844‑0
    https://doi.org/10.1007/978-94-010-0844-0 [Google Scholar]
  3. Baroni, M. & Evert, S.
    2008 “Statistical methods for corpus exploitation”. In A. Lüdeling & M. Kytö (Eds.), Corpus Linguistics. An International Handbook. Berlin: Mouton de Gruyter, 777–803.
    [Google Scholar]
  4. Biber, D.
    1988Variation across Speech and Writing. Cambridge: Cambridge University Press. doi: 10.1017/CBO9780511621024
    https://doi.org/10.1017/CBO9780511621024 [Google Scholar]
  5. Biber, D. & Reppen, R.
    2015a “Introduction”. In D. Biber & R. Reppen (Eds.), The Cambridge Handbook of Corpus Linguistics. Cambridge: Cambridge University Press, 1–8. doi: 10.1017/CBO9781139764377.001
    https://doi.org/10.1017/CBO9781139764377.001 [Google Scholar]
  6. (Eds.) 2015bThe Cambridge Handbook of Corpus Linguistics. Cambridge: Cambridge University Press. doi: 10.1017/CBO9781139764377
    https://doi.org/10.1017/CBO9781139764377 [Google Scholar]
  7. Bley-Vroman, R.
    1988 “The fundamental character of foreign language learning”. In W. Rutherford & M. Sharwood Smith (Eds.), Grammar and Second Language Teaching: A Book of Readings. Rowley, MA: Newbury House, 19–30.
    [Google Scholar]
  8. Boyd, A. , Hana, J. , Nicolas, L. , Meurers, D. , Wisniewski, K. , Abel, A. , Schöne, K. , Štindlová, B. & Vettori, C.
    2014 “The MERLIN corpus: Learner language and the CEFR”. Proceedings of theNinth International Conference on Language Resources and Evaluation (LREC’14), European Language Resources Association (ELRA), Reykjavik, May 26–31, 2014.
    [Google Scholar]
  9. Brezina, V. & Meyerhoff, M.
    2014 “Significant or random? A critical review of sociolinguistic generalisations based on large corpora”, International Journal of Corpus Linguistics19(1), 1–28. doi: 10.1075/ijcl.19.1.01bre
    https://doi.org/10.1075/ijcl.19.1.01bre [Google Scholar]
  10. Byrnes, H.
    2013 “Notes from the editor”, The Modern Language Journal97(4), 825–827. doi: 10.1111/j.1540‑4781.2013.12051.x
    https://doi.org/10.1111/j.1540-4781.2013.12051.x [Google Scholar]
  11. Carlsen, C.
    2012 “Proficiency level – a fuzzy variable in computer learner corpora”, Applied Linguistics33(2), 161–183. doi: 10.1093/applin/amr047
    https://doi.org/10.1093/applin/amr047 [Google Scholar]
  12. Dagneaux, E. , Denness, S. & Granger, S.
    1998 “Computer-aided error analysis”, System: An International Journal of Educational Technology and Applied Linguistics26(2), 163–174. doi: 10.1016/S0346‑251X(98)00001‑3
    https://doi.org/10.1016/S0346-251X(98)00001-3 [Google Scholar]
  13. DeKeyser, R. , Alfi-Shabtay, I. & Ravid, D.
    2010 “Cross-linguistic evidence for the nature of age effects in second language acquisition”, Applied Psycholinguistics31, 413–438. doi: 10.1017/S0142716410000056
    https://doi.org/10.1017/S0142716410000056 [Google Scholar]
  14. Derrick, D. J.
    2016 “Instrument reporting practices in second language research”, TESOL Quarterly50, 132–153.
    [Google Scholar]
  15. dissertation.laerd.com
    dissertation.laerd.com 2012: online. “Types of quantitative research question”, Laerd Dissertation Online. Available at: dissertation.laerd.com/types-of-quantitative-research-question.php (accessedMay 2016).
    [Google Scholar]
  16. Durrant, P.
    2014 “Corpus frequency and second language learners’ knowledge of collocations: A meta-analysis”, International Journal of Corpus Linguistics19, 443–477. doi: 10.1075/ijcl.19.4.01dur
    https://doi.org/10.1075/ijcl.19.4.01dur [Google Scholar]
  17. Gass, S.
    2009 “A historical survey of SLA research”. In T. K. Bhatia & W. C. Ritchie (Eds.), The New Handbook of Second Language Acquisition. Bingley, England: Emerald, 3–27.
    [Google Scholar]
  18. Gass, S. , Fleck, C. , Leder, N. , & Svetics, I.
    1998 “Ahistoricity revisited. Does SLA have a history?”, Studies in Second Language Acquisition20, 407–421.
    [Google Scholar]
  19. Gelman, A. & Weakliem, D.
    2009 “Of beauty, sex, and power: Too little attention has been paid to the statistical challenges in estimating small effects”, American Scientist97, 310–316. doi: 10.1511/2009.79.310
    https://doi.org/10.1511/2009.79.310 [Google Scholar]
  20. Gilquin, G. , Granger, S. & Paquot, M.
    2007 “Learner corpora: The missing link in EAP pedagogy”, Journal of English for Academic Purposes6(4), 319–335. doi: 10.1016/j.jeap.2007.09.007
    https://doi.org/10.1016/j.jeap.2007.09.007 [Google Scholar]
  21. Gilquin, G. , De Cock, S. & Granger, S.
    2010The Louvain International Database of Spoken English Interlanguage. Handbook and CD-ROM. Louvain-La-Neuve: Presses universitaires de Louvain.
    [Google Scholar]
  22. Godfroid, A. & Spino, L.
    2015 “Reconceptualizing reactivity of think-alouds and eye tracking: Absence of evidence is not evidence of absence”, Language Learning65, 896–928. doi: 10.1111/lang.12136
    https://doi.org/10.1111/lang.12136 [Google Scholar]
  23. Granger, S.
    1996 “From CA to CIA and back: an integrated approach to computerized bilingual and learner corpora”. In K. Aijmer , B. Altenberg , & M. Johansson (Eds.), Languages in Contrast. Text-based Cross-linguistic Studies. Lund: Lund University Press, 37–51.
    [Google Scholar]
  24. 1997 “Automated retrieval of passives from native and learner corpora: precision and recall”, Journal of English Linguistics25(4), 365–374. doi: 10.1177/007542429702500410
    https://doi.org/10.1177/007542429702500410 [Google Scholar]
  25. 2004 “Computer learner corpus research: Current status and future prospects”. In U. Connor & T. Upton (Eds.), Applied Corpus Linguistics: A Multidimensional Perspective. Amsterdam and Atlanta: Rodopi, 123–145.
    [Google Scholar]
  26. 2009 “The contribution of learner corpora to second language acquisition and foreign language teaching: A critical evaluation”. In K. Aijmer (Ed.), Corpora and Language Teaching. Amsterdam: John Benjamins, 13–32. doi: 10.1075/scl.33.04gra
    https://doi.org/10.1075/scl.33.04gra [Google Scholar]
  27. 2015 “Contrastive Interlanguage Analysis: A reappraisal”, International Journal of Learner Corpus Research1(1): 7–24. doi: 10.1075/ijlcr.1.1.01gra
    https://doi.org/10.1075/ijlcr.1.1.01gra [Google Scholar]
  28. Granger, S. & Bestgen, Y.
    forthcoming. “Using collgrams to assess L2 phraseological development: A replication study”. In P. de Haan , R. de Vries , & S. van Vuuren Eds. Language, Learners and Levels: Progression and Variation. Louvain-la-Neuve: Presses universitaires de Louvain.
    [Google Scholar]
  29. Granger, S. , Dagneaux, E. , Meunier, F. & Paquot, M.
    2009International Corpus of Learner English. Version 2 (Handbook + CD-Rom). Louvain-la-Neuve: Presses universitaires de Louvain.
    [Google Scholar]
  30. Granger, S. , Gilquin, G. & Meunier, F.
    2015The Cambridge Handbook of Learner Corpus Research. Cambridge: Cambridge University Press. doi: 10.1017/CBO9781139649414
    https://doi.org/10.1017/CBO9781139649414 [Google Scholar]
  31. Gries S.
    2006a “Some proposals towards more rigorous corpus linguistics”, Zeitschrift für Anglistik und Amerikanistik54(2), 191–202. doi: 10.1515/zaa‑2006‑0209
    https://doi.org/10.1515/zaa-2006-0209 [Google Scholar]
  32. 2006b “Exploring variability within and between corpora: Some methodological considerations”, Corpora1(2), 109–151. doi: 10.3366/cor.2006.1.2.109
    https://doi.org/10.3366/cor.2006.1.2.109 [Google Scholar]
  33. Gries, S.
    2010 “Methodological skills in corpus linguistics: a polemic and some pointers towards quantitative methods”. In T. Harris & M. Moreno Jaén (Eds.), Corpus Linguistics in Language Teaching. Frankfurt: Peter Lang, 121–146.
    [Google Scholar]
  34. 2013a “Statistical tests for the analysis of learner corpus data”. In A. Diaz-Negrillo , N. Ballier , & P. Thompson (Eds.), Automatic Treatment and Analysis of Learner Corpus Data. Amsterdam and Philadelphia: Benjamins. doi: 10.1075/scl.59.17gri
    https://doi.org/10.1075/scl.59.17gri [Google Scholar]
  35. 2013bStatistics for Linguistics with R. A Practical Introduction (2nd edition). Berlin and Boston: De Gruyter Mouton. doi: 10.1515/9783110307474
    https://doi.org/10.1515/9783110307474 [Google Scholar]
  36. 2014 “Quantitative corpus approaches to linguistic analysis: seven or eight levels of resolution and the lessons they teach us”. In I. Taavitsainen , M. Kytö , C. Claridge , & J. Smith (Eds.), Developments in English: Expanding Electronic Evidence. Cambridge: Cambridge University Press, 29–47. doi: 10.1017/CBO9781139833882.005
    https://doi.org/10.1017/CBO9781139833882.005 [Google Scholar]
  37. 2015a “Statistics for learner corpus research”. In S. Granger , G. Gilquin , & F. Meunier (Eds.), The Cambridge Handbook of Learner Corpus Research. Cambridge: Cambridge University Press, 159–181. doi: 10.1017/CBO9781139649414.008
    https://doi.org/10.1017/CBO9781139649414.008 [Google Scholar]
  38. 2015b “Quantitative designs and statistical techniques”. In D. Biber & R. Reppen (Eds.), The Cambridge Handbook of English Corpus Linguistics. Cambridge: Cambridge University Press, 50–71. doi: 10.1017/CBO9781139764377.004
    https://doi.org/10.1017/CBO9781139764377.004 [Google Scholar]
  39. 2015c “Some current quantitative problems in corpus linguistics and a sketch of some solutions”, Language and Linguistics16(1), 93–117. doi: 10.1177/1606822X14556606
    https://doi.org/10.1177/1606822X14556606 [Google Scholar]
  40. Gries, S. & Wulff, S.
    2013 “The genitive alternation in Chinese and German ESL learners. Towards a multifactorial notion of context in learner corpus research”, International Journal of Corpus Linguistics18(3), 327–356. doi: 10.1075/ijcl.18.3.04gri
    https://doi.org/10.1075/ijcl.18.3.04gri [Google Scholar]
  41. Hulstijn, J. , Alderson, C. & Schroonen, R.
    2010 “Developmental stages in second language acquisition and levels of second-language proficiency: Are there links between them?”. In I. Bartning , M. Martin , & I. Vedder (Eds.), Communicative Proficiency and Linguistic Development: Intersections between SLA and language Testing Research. European Second Language Assocation: EUROSLA Series Monographs 1, 11–22.
    [Google Scholar]
  42. Ioannidis J. P. A. , Fanelli, D. , Dunne, D. D. & Goodman, S. N.
    2015 “Meta-research: Evaluation and improvement of research methods and practices”, PLoS Biology13(10), 1–7. doi: 10.1371/journal.pbio.1002264
    https://doi.org/10.1371/journal.pbio.1002264 [Google Scholar]
  43. Kilgarriff, A.
    2001 “Comparing corpora”, International Journal of Corpus Linguistics6(1), 97–133. doi: 10.1075/ijcl.6.1.05kil
    https://doi.org/10.1075/ijcl.6.1.05kil [Google Scholar]
  44. 2005 “Language is never, ever, ever random”, Corpus Linguistics and Linguistic Theory1–2, 263–275.
    [Google Scholar]
  45. Köhler, R.
    2013 “Statistical Comparability: Methodological Caveats”. In S. Sharoff , R. Rapp , P. Zweigenbaum , & P. Fung (Eds.), Building and Using Comparable Corpora. Berlin and Heidelberg: Springer, 77–91. doi: 10.1007/978‑3‑642‑20128‑8_4
    https://doi.org/10.1007/978-3-642-20128-8_4 [Google Scholar]
  46. Larson-Hall, J. & Herrington, R.
    2010 “Improving data analysis in second language acquisition by utilizing modern developments in applied statistics”, Applied Linguistics31(3), 368–390. doi: 10.1093/applin/amp038
    https://doi.org/10.1093/applin/amp038 [Google Scholar]
  47. Larson-Hall, J. & Plonsky, L.
    2015 “Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field”, Language Learning65(S1), 127–159. doi: 10.1111/lang.12115
    https://doi.org/10.1111/lang.12115 [Google Scholar]
  48. Levshina, N.
    2015How to Do Linguistics with R: Data Exploration and Statistical Analysis. Amsterdam and Philadelphia: John Benjamins. doi: 10.1075/z.195
    https://doi.org/10.1075/z.195 [Google Scholar]
  49. Lipsey, M. W.
    2009 “Identifying interesting variables and analysis opportunities”. In H. Cooper , L. V. Hedges , & J. C. Valentine (Eds.), The Handbook of Research Synthesis (2nd edition). New York: Russell Sage Foundation, 147–158.
    [Google Scholar]
  50. Liu, Q. & Brown, D.
    2015 “Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing”, Journal of Second Language Writing30, 66–81. doi: 10.1016/j.jslw.2015.08.011
    https://doi.org/10.1016/j.jslw.2015.08.011 [Google Scholar]
  51. Mahboob, A. , Paltridge, B. , Phakiti, A. , Wagner, E. , Starfield, S. , Burns, A. , Jones, R. H. & De Costa, P. I.
    2016 “TESOL Quarterly Research Guidelines”, TESOL Quarterly50, 42–65.
    [Google Scholar]
  52. Mackey, A. & Gass, S.
    2011Research Methods in Second Language Acquisition: A Practical Guide. Oxford: Wiley-Blackwell. doi: 10.1002/9781444347340
    https://doi.org/10.1002/9781444347340 [Google Scholar]
  53. Mackey, A. & Marsden, E.
    (Eds.) 2016Advancing Methodology and Practice: The IRIS Repository of Instruments for Research into Second Languages. New York: Routledge.
    [Google Scholar]
  54. Marsden, E. , Mackey, A. & Plonsky, L.
    2016 “Breadth and depth: The IRIS repository”. In A. Mackey & E. Marsden (Eds.), Advancing Methodology and Practice: The IRIS Repository of Instruments for Research into Second Languages. New York: Routledge, 1–21.
    [Google Scholar]
  55. Meunier, F.
    2010 “Learner corpora and English language teaching: Checkup time”, Anglistik: International Journal of English Studies21(1), 209–220.
    [Google Scholar]
  56. Myles, F.
    2008 “Investigating learner language development with electronic longitudinal corpora: Theoretical and methodological issues”. In L. Ortega & H. Byrnes (Eds.), The Longitudinal Study of Advanced L2 Capacities. New York and London: Routledge, 58–72.
    [Google Scholar]
  57. Norris, J.
    2015 “Statistical significance testing in second language research: Basic problems and suggestions for reform”, Language Learning65(S1), 97–126. doi: 10.1111/lang.12114
    https://doi.org/10.1111/lang.12114 [Google Scholar]
  58. Norris, J. , Plonsky, L. , Ross, S. & R. Schoonen
    2015 “Guidelines for reporting quantitative methods and results in primary research”, Language Learning65(2): 470–476. doi: 10.1111/lang.12104
    https://doi.org/10.1111/lang.12104 [Google Scholar]
  59. Ortega, L.
    2009Understanding Second Language Acquisition. London: Hodder Education.
    [Google Scholar]
  60. 2013 “SLA for the 21st century: Disciplinary progress, transdisciplinary relevance, and the bi/multilingual turn”, Language Learning63(S1), 1–24. doi: 10.1111/j.1467‑9922.2012.00735.x
    https://doi.org/10.1111/j.1467-9922.2012.00735.x [Google Scholar]
  61. Ortega, L. & Iberri-Shea, G.
    2005 “Longitudinal research in second language acquisition: recent trends and future directions”, Annual Review of Applied Linguistics25, 26–45. doi: 10.1017/S0267190505000024
    https://doi.org/10.1017/S0267190505000024 [Google Scholar]
  62. Ortega, L. & Byrnes, H.
    2008 “Theorizing advancedness, setting up the longitudinal research agenda”. In L. Ortega & H. Byrnes (Eds.), The Longitudinal Study of Advanced L2 Capacities. New York: Routledge, 281–300.
    [Google Scholar]
  63. Paquot, M. & Bestgen, Y.
    2009 “Distinctive words in academic writing: a comparison of three statistical tests for keyword extraction”. In A. Jucker , D. Schreier & M. Hundt (Eds.), Corpora: Pragmatics and Discourse. Papers from the 29th International Conference on English Language Research on Computerized Corpora (ICAME 29). Amsterdam: Rodopi, 247–269.
    [Google Scholar]
  64. Paquot, M. & Granger, S.
    2012 “Formulaic Language in Learner Corpora”, Annual Review of Applied Linguistics32, 130–149. doi: 10.1017/S0267190512000098
    https://doi.org/10.1017/S0267190512000098 [Google Scholar]
  65. Pendar, N. & Chapelle, C.
    2008 “Investigating the promise of learner corpora: Methodological issues”, CALICO Journal25(2), 189–206.
    [Google Scholar]
  66. Plonsky, L.
    2013 “Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research”, Studies in Second Language Acquisition35, 655–687. doi: 10.1017/S0272263113000399
    https://doi.org/10.1017/S0272263113000399 [Google Scholar]
  67. 2014 “Study quality in quantitative L2 research (1990–2010): A methodological synthesis and call for reform”, The Modern Language Journal98(1), 450–470. doi: 10.1111/j.1540‑4781.2014.12058.x
    https://doi.org/10.1111/j.1540-4781.2014.12058.x [Google Scholar]
  68. 2015a “Statistical power, p values, descriptive statistics, and effect sizes: a ‘back-to-basics’ approach to advancing quantitative methods in L2 research”. In L. Plonsky (Ed.), Advancing Quantitative Methods in Second Language Research. New York: Routledge, 23–45.
    [Google Scholar]
  69. (Ed.) 2015bAdvancing Quantitative Methods in Second Language Research. New York: Routledge.
    [Google Scholar]
  70. 2015c “Quantitative considerations for improving replicability in CALL and applied linguistics”, CALICO Journal32, 232–244.
    [Google Scholar]
  71. Plonsky, L. & Derrick, D. J.
    2016 “A meta-analysis of reliability coefficients in second language research”, Modern Language Journal100, 538–558.
    [Google Scholar]
  72. Plonsky, L. & Gass, S.
    2011 “Quantitative research methods, study quality, and outcomes: The case of interaction research”, Language Learning61, 325–366. doi: 10.1111/j.1467‑9922.2011.00640.x
    https://doi.org/10.1111/j.1467-9922.2011.00640.x [Google Scholar]
  73. Plonsky, L. , Egbert, J. & LaFlair, G. T.
    2015 “Bootstrapping in applied linguistics: Assessing its potential using shared data”, Applied Linguistics36, 591–610.
    [Google Scholar]
  74. Plonsky, L. & Kim, Y.
    2016 “Task-based learner production: A substantive and methodological review”, Annual Review of Applied Linguistics36, 73–97.
    [Google Scholar]
  75. Plonsky, L. & Oswald, F. L.
    2015 “Meta-analyzing second language research”. In L. Plonsky (Ed.), Advancing Quantitative Methods in Second Language Research. New York: Routledge, 106–128.
    [Google Scholar]
  76. in press. Multiple regression as a flexible alternative to ANOVA in L2 research. Studies in Second Language Acquisition. doi: 10.1017/S0272263116000231
    https://doi.org/10.1017/S0272263116000231 [Google Scholar]
  77. Porte, G.
    (Ed.) 2012Replication Research in Applied Linguistics. Cambridge: Cambridge University Press.
    [Google Scholar]
  78. Present-Thomas, R. L. , Weltens, B. & de Jong, J. H. A. L.
    2013 “Defining proficiency: A comparative analysis of CEF level classification methods in a written learner corpus”, Dutch Journal of Applied Linguistics2(1), 57–76. doi: 10.1075/dujal.2.1.07pre
    https://doi.org/10.1075/dujal.2.1.07pre [Google Scholar]
  79. Römer, U.
    2009 “The inseparability of lexis and grammar. Corpus linguistic perspectives”, Annual Review of Cognitive Linguistics7, 141–163. doi: 10.1075/arcl.7.06rom
    https://doi.org/10.1075/arcl.7.06rom [Google Scholar]
  80. Ross, S. & Mackey, B.
    2015 “Bayesian Approaches to Imputation, Hypothesis Testing, and Parameter Estimation”, Language Learning65 (Supp.1), 208–227. doi: 10.1111/lang.12118
    https://doi.org/10.1111/lang.12118 [Google Scholar]
  81. Selinker, L.
    1972 “Interlanguage”, International Review of Applied Linguistics in Language Teaching10, 209–231. doi: 10.1515/iral.1972.10.1‑4.209
    https://doi.org/10.1515/iral.1972.10.1-4.209 [Google Scholar]
  82. Sinclair, J. McH.
    1991Corpus, Concordance, Collocation. Oxford: Oxford University Press.
    [Google Scholar]
  83. Thomas, M.
    2006 “Research synthesis and historiography: The case of assessment of second language proficiency”. In J. M. Norris & L. Ortega (Eds.), Synthesizing Research on Language Learning and Teaching. Philadelphia, PA: John Benjamins, 279–298. doi: 10.1075/lllt.13.13tho
    https://doi.org/10.1075/lllt.13.13tho [Google Scholar]
  84. Wells, K. & Littell, J. H.
    2009 “Study quality assessment in systematic reviews of research on intervention effects”, Research on Social Work Practice19, 52–62. doi: 10.1177/1049731508317278
    https://doi.org/10.1177/1049731508317278 [Google Scholar]
  85. Ziegler, N.
    2016 “Methodological practices in interaction in synchronous computer mediated communication: A synthetic approach”. In A. Mackey & E. Marsden (Eds.), Instruments for Research into Second Languages: Empirical Studies Advancing Methodology. New York: Routledge, 197–223.
    [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error