1887
  1. Home
  2. e-Journals
  3. International Journal of Learner Corpus Research
  4. Volume 9, Issue 2
  5. Article
Volume 9, Issue 2
  • ISSN 2215-1478
  • E-ISSN: 2215-1486
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This paper introduces the open-source English Language Learning Insight, Proficiency and Skills Evaluation (ELLIPSE) corpus. The corpus comprises ~6,500 essays written by English language learners (ELLs). All essays were written during state-wide standardized annual testing in the United States. The essays were written on 29 different independent prompts that required no background knowledge on the part of the writer. Individual difference information is made available for each essay including economic status, gender, grade level (8–12), and race/ethnicity. Each essay was scored by two trained human raters for English language proficiency including an overall score of English proficiency and analytic scores for cohesion, syntax, vocabulary, phraseology, grammar, and conventions. The paper provides reliability on the human judgments of proficiency reported for the corpus. The ELLIPSE corpus addresses many of the concerns found in existing learner corpora including unique holistic and analytic scores for each ELL essay. The corpus also includes limited demographic and individual difference data for each ELL.

© 2023 John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/ijlcr.22026.cro
2024-02-08
2024-12-08

  • From This Site

    /content/journals/10.1075/ijlcr.22026.cro
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Bachman, L. F., & Palmer, A. S.
    (1996) Language testing in practice: Designing and developing useful language tests (Vol.11). Oxford University Press.
     [Google Scholar]
  2. Bailey, A. L., & Kelly, K. R.
    (2010) The use and validity of home language surveys in state English language proficiency assessment systems: A review and issues perspective (Evaluating the Validity of English Language Proficiency Assessment). edCount, LLC Center of Assessment UCLA. https://edcount.com/wp-content/uploads/2020/10/Bailey-Kelly-HLS-EVEA-white-paper-July-2010-revised-Jan-2011and-Sept-2011.pdf
     [Google Scholar]
  3. Birdsong, D.
    (2005) Interpreting age effects in second language acquisition. InJ. F. Kroll & A. M. B. de Groot (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp.109–127). Oxford University Press.
     [Google Scholar]
  4. Blanchard, D., Tetreault, J., Higgins, D., Cahill, A., & Chodorow, M.
    (2013) TOEFL11: A corpus of non-native English. ETS Research Report Series 2013(2).
     [Google Scholar]
  5. Boyd, A., Hana, J., Nicolas, L., Meurers, D., Wisniewski, K., Abel, A., Schöne, K., Štindlová, B., & Vettori, C.
    (2014) The MERLIN corpus: Learner language and the CEFR. InN. Calzolari, K. Choukri, T. Declerck, H. Loftsson, B. Maegaard, J. Mariani, A. Moreno, J. Odijk, & S. Piperidis (Eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14) (pp: 1281–1288). European Language Resources Association (ELRA).
     [Google Scholar]
  6. Chapelle, C. A., Enright, M. K., & Jamieson, J.
    (Eds.) (2008) Building a validity argument for the Test of English as a Foreign Language. Routledge.
     [Google Scholar]
  7. Cheng, W., & Warren, M.
    (2005) Peer assessment of language proficiency. Language Testing, 22(1), 93–121. 10.1191/0265532205lt298oa
     https://doi.org/10.1191/0265532205lt298oa [Google Scholar]
  8. Choi, I.
    (2016) Efficacy of an ICALL tutoring system and process-oriented corrective feedback. Computer Assisted Language Learning, 29(2), 334–364. 10.1080/09588221.2014.960941
     https://doi.org/10.1080/09588221.2014.960941 [Google Scholar]
  9. Chomsky, C.
    (1972) Stages in Language Development and Reading Exposure. Harvard Educational Review, 42(1), 1–33. 10.17763/haer.42.1.h78l676h28331480
     https://doi.org/10.17763/haer.42.1.h78l676h28331480 [Google Scholar]
  10. Clifford, R., & Cox, T. L.
    (2013) Empirical validation of reading proficiency guidelines. Foreign Language Annals, 46(1), 45–61. 10.1111/flan.12033
     https://doi.org/10.1111/flan.12033 [Google Scholar]
  11. Cohen, J.
    (1992) Statistical Power Analysis. Current Directions in Psychological Science, 1(3), 98–101. 10.1111/1467‑8721.ep10768783
     https://doi.org/10.1111/1467-8721.ep10768783 [Google Scholar]
  12. Council of Europe
    Council of Europe (2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
     [Google Scholar]
  13. Crossley, S. A., Kyle, K., & McNamara, D. S.
    (2016) The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods, 48(4), 1227–1237. 10.3758/s13428‑015‑0651‑7
     https://doi.org/10.3758/s13428-015-0651-7 [Google Scholar]
  14. (2017) Sentiment Analysis and Social Cognition Engine (SEANCE): An automatic tool for sentiment, social cognition, and social-order analysis. Behavior Research Methods, 49(3), 803–821. 10.3758/s13428‑016‑0743‑z
     https://doi.org/10.3758/s13428-016-0743-z [Google Scholar]
  15. Crossley, S. A., & McNamara, D. S.
    (2010) Cohesion, coherence, and expert evaluations of writing proficiency. InS. Ohlsson & R. Catrambone (Eds.), Proceedings of the Annual Meeting of the Cognitive Science Society (pp.984–989). Cognitive Science Society.
     [Google Scholar]
  16. Crossley, S., Salsbury, T., McNamara, D. S.
    (2013) Validating lexical measures using human scores of lexical proficiency. InS. Jarvis & M. Daller (Eds.), Vocabulary knowledge: Human ratings and automated measures (pp.105–134). John Benjamins. 10.1075/sibil.47.06ch4
     https://doi.org/10.1075/sibil.47.06ch4 [Google Scholar]
  17. Ellis, R.
    (1991) Grammatically judgments and second language acquisition. Studies in Second Language Acquisition, 13(2), 161–186. 10.1017/S0272263100009931
     https://doi.org/10.1017/S0272263100009931 [Google Scholar]
  18. (2003) Task-based language learning and teaching. Oxford University Press.
     [Google Scholar]
  19. Figueras, N.
    (2012) The impact of the CEFR. ELT Journal, 66(4), 477–485. 10.1093/elt/ccs037
     https://doi.org/10.1093/elt/ccs037 [Google Scholar]
  20. Foddy, W.
    (1993) Constructing questions for interviews and questionnaires: Theory and practice in social research. Cambridge University Press. 10.1017/CBO9780511518201
     https://doi.org/10.1017/CBO9780511518201 [Google Scholar]
  21. Geertzen, J., Alexopoulou, T., & Korhonen, A.
    (2013) Automatic linguistic annotation of large scale L2 databases: The EF-Cambridge Open Language Database (EFCAMDAT). InR. T. Miller, K. I. Martin, C. M. Eddington, A. Henery, N. Marcos Miguel, A. M. Tseng, A. Tuninetti, & D. Walter (Eds.), Proceedings of the 31st Second Language Research Forum: Building Bridges Between Disciplines (pp.240–254). Cascadilla Proceedings Project.
     [Google Scholar]
  22. Granena, G.
    (2019) Cognitive aptitudes and L2 speaking proficiency: Links between LLAMA and Hi-LAB. Studies in Second Language Acquisition, 41(2), 313–336. 10.1017/S0272263118000256
     https://doi.org/10.1017/S0272263118000256 [Google Scholar]
  23. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z.
    (2004) Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202. 10.3758/BF03195564
     https://doi.org/10.3758/BF03195564 [Google Scholar]
  24. Housen, A., & Kuiken, F.
    (2009) Complexity, accuracy, and fluency in second language acquisition. Applied linguistics, 30(4), 461–473. 10.1093/applin/amp048
     https://doi.org/10.1093/applin/amp048 [Google Scholar]
  25. Housen, A., Kuiken, F., & Vedder, I.
    (Eds.) (2012) Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (Vol.321). John Benjamins. 10.1075/lllt.32
     https://doi.org/10.1075/lllt.32 [Google Scholar]
  26. Hymes, D.
    (1972) Editorial Introduction to Language in Society. Language in Society, 1(1), 1–14. 10.1017/S0047404500006515
     https://doi.org/10.1017/S0047404500006515 [Google Scholar]
  27. Ishikawa, S. I.
    (2013) The ICNALE and sophisticated contrastive interlanguage analysis of Asian learners of English. Learner Corpus Studies in Asia and the World, 1(1), 91–118.
     [Google Scholar]
  28. Kim, A. Y.
    (2015) Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability. Language Testing, 32(2), 227–258. 10.1177/0265532214558457
     https://doi.org/10.1177/0265532214558457 [Google Scholar]
  29. Kyle, K., & Crossley, S. A.
    (2018) Measuring Syntactic Complexity in L2 Writing Using Fine-Grained Clausal and Phrasal Indices. The Modern Language Journal, 102(2), 333–349. 10.1111/modl.12468
     https://doi.org/10.1111/modl.12468 [Google Scholar]
  30. Kyle, K., Crossley, S., & Berger, C.
    (2018) The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods, 50(3), 1030–1046. 10.3758/s13428‑017‑0924‑4
     https://doi.org/10.3758/s13428-017-0924-4 [Google Scholar]
  31. Kyle, K., Crossley, S. A., & Jarvis, S.
    (2021) Assessing the Validity of Lexical Diversity Indices Using Direct Judgements. Language Assessment Quarterly, 18(2), 154–170. 10.1080/15434303.2020.1844205
     https://doi.org/10.1080/15434303.2020.1844205 [Google Scholar]
  32. Lagakis, P., & Demetriadis, S.
    (2021) Automated essay scoring: A review of the field. 2021 International Conference on Computer, Information and Telecommunication Systems (CITS), 1–6. 10.1109/CITS52676.2021.9618476
     https://doi.org/10.1109/CITS52676.2021.9618476 [Google Scholar]
  33. Larsen-Freeman, D.
    (1978) An ESL Index of Development. TESOL Quarterly, 12(4), 439–448. 10.2307/3586142
     https://doi.org/10.2307/3586142 [Google Scholar]
  34. Laufer, B., & Nation, P.
    (1999) A vocabulary-size test of controlled productive ability. Language Testing, 16(1), 33–51. 10.1177/026553229901600103
     https://doi.org/10.1177/026553229901600103 [Google Scholar]
  35. Lim, G. S.
    (2011) The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced raters. Language Testing, 28(4), 543–560. 10.1177/0265532211406422
     https://doi.org/10.1177/0265532211406422 [Google Scholar]
  36. Linacre, J. M.
    (2021) A User’s Guide to FACETS Rasch-Model Computer Programs. Program Manual3.83.5.
     [Google Scholar]
  37. Lisken-Gasparro, J. E.
    (1984) The ACTFL proficiency guidelines: Gateway to testing and curriculum. Foreign Language Annals17(5), 475–489. 10.1111/j.1944‑9720.1984.tb01736.x
     https://doi.org/10.1111/j.1944-9720.1984.tb01736.x [Google Scholar]
  38. Lumley, T.
    (1998) Perceptions of language-trained raters and occupational experts in a test of occupational English language profficiency. English for Specific Purposes, 17(4), 347–367. 10.1016/S0889‑4906(97)00016‑1
     https://doi.org/10.1016/S0889-4906(97)00016-1 [Google Scholar]
  39. (2005) Assessing second language writing: The rater’s perspective. Peter Lang.
     [Google Scholar]
  40. Meurers, D., De Kuthy, K., Nuxoll, F., Rudzewitz, B., & Ziai, R.
    (2019) Scaling up intervention studies to investigate real-life foreign language learning in school. Annual Review of Applied Linguistics, 391, 161–188. 10.1017/S0267190519000126
     https://doi.org/10.1017/S0267190519000126 [Google Scholar]
  41. McNamara, T., Knoch, U., Fan, J., & Rossner, R.
    (2019) Fairness, justice & language assessment. Oxford University Press.
     [Google Scholar]
  42. Ortega, L.
    (2012) Epilogue: Exploring L2 writing–SLA interfaces. Journal of Second Language Writing, 21(4), 404–415. 10.1016/j.jslw.2012.09.002
     https://doi.org/10.1016/j.jslw.2012.09.002 [Google Scholar]
  43. O’Sullivan, B.
    (2018) IELTS (international English language testing system). InJ. I. Liontas (Ed. in Chief), The TESOL Encyclopedia of English Language Teaching (pp.1–8). Wiley. 10.1002/9781118784235.eelt0359
     https://doi.org/10.1002/9781118784235.eelt0359 [Google Scholar]
  44. Plonsky, L.
    (2023) Sampling and Generalizability in Lx Research: A Second-Order Synthesis. Languages8(1), 751, 1–13. 10.3390/languages8010075
     https://doi.org/10.3390/languages8010075 [Google Scholar]
  45. Skehan, P.
    (1989) Individual differences in second-language learning. Edward Arnold.
     [Google Scholar]
  46. U.S. Department of Education
    U.S. Department of Education (2017) Our nation’s English learners. US Department of Education. https://www2.ed.gov/datastory/el-characteristics/index.html
     [Google Scholar]
  47. Weigle, S. C.
    (2004) Integrating reading and writing in a competency test for non-native speakers of English. Assessing Writing, 9(1), 27–55. 10.1016/j.asw.2004.01.002
     https://doi.org/10.1016/j.asw.2004.01.002 [Google Scholar]
  48. Widdowson, H. G.
    (1983) Learning purpose and language use. Oxford University Press.
     [Google Scholar]
  49. Wood, C., & Schatschneider, C.
    (2021) Examining Writing Measures and Achievement for Students of Varied Language Abilities and Linguistic. Reading and Writing Quarterly, 37(1), 65–81. 10.1080/10573569.2020.1716284
     https://doi.org/10.1080/10573569.2020.1716284 [Google Scholar]
/content/journals/10.1075/ijlcr.22026.cro
Loading
The English Language Learner Insight, Proficiency and Skills Evaluation (ELLIPSE) Corpus
International Journal of Learner Corpus Research 9, 248 (2023); https://doi.org/10.1075/ijlcr.22026.cro
/content/journals/10.1075/ijlcr.22026.cro
/content/journals/10.1075/ijlcr.22026.cro
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): language proficiency; language testing; open source learner corpus

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error