1887
  1. Home
  2. e-Journals
  3. ITL - International Journal of Applied Linguistics
  4. Volume 127, Issue 1
  5. Article
Volume 127, Issue 1
  • ISSN 0019-0829
  • E-ISSN: 1783-1490
USD
Buy:$35.00 + Taxes

Abstract

This article examines the relationship between two kinds of methods used to assess the quality of second language writing : 1) objective computerised text analysis focusing on the linguistic features of written texts, and 2) subjective evaluation performed by human raters using a combination of holistic and analytical scoring procedures. In particular, it attempts to explore the potentials and possible limitations of using computerised programs as research tools in second language writing research.

The written sample consisted of a total of 132 short essays written b y ESL students enrolled in various academic programs at an American university. The first method used computerized programs to assess the written texts in terms of syntactic complexity, lexical complexity and grammatical accuracy, whereas in the second method, two ESL raters evaluated the same sample of texts by first assigning a holistic score to each piece of writing, then applying an analytical scheme to assess linguistic features at the syntactic, lexical and grammatical level as well as textual and rhetorical features at the discourse level. A series of correlation analyses were performed using the scores obtained from these two kinds of assessment procedures at the correspondent levels. The results show that a significant correlation was consistently found between these two kinds of scores at the level of grammatical accuracy, yet n o significant correlation was found in any of the other categories. The results also indicate a high level of internal consistency in t he computerized analysis.

© 2000 John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/itl.127-128.02li
2000-01-01
2025-04-20

  • From This Site

    /content/journals/10.1075/itl.127-128.02li
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. BIBER, D.
    (1988) : Variations across speech and writing. Cambridge : Cambridge University Press.
     [Google Scholar]
  2. (1996) : Corpus-based investigation of language use. Annual Review of Applied Linguistics16 : 115–136.
     [Google Scholar]
  3. COHEN, A.D.
    (1994) : Assessing language ability in the classroom. Boston, Mass : Heinle & Heinle Publishers.
     [Google Scholar]
  4. CONNOR-LlNTON, J.
    (1995): Looking behind the curtain : What do L2 composition ratings really mean? TESOL Quarterly29, 4 : 762–765.
     [Google Scholar]
  5. CONNOR, U.
    (1991): Linguistic/rhetorical measures for evaluating ESL writing. In L. Hamp-Lyons (Ed.), assessing second language writing in academic contexts (pp.215–25). Norwood, NJ : Ablex.
     [Google Scholar]
  6. ELBOW, P.
    (1996) : Writing assessment in the 21st century : A Utopian view. In L.Z. Bloom , D.A. Daiker , and E.M. White (eds) Composition in the 21st century : Crisis and change (pp.83–100). Carbon-dale : Southern Illinois University Press.
     [Google Scholar]
  7. FERRIS, D.
    (1993) : The design of an automatic analysis program for L2 text research : Necessity and feasibility. Journal of Second Language Writing2, 2:119–129.
     [Google Scholar]
  8. (1994): Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28, 2:414–20.
     [Google Scholar]
  9. HAMP-LYONS, L.
    (1991) : Scoring procedures for ESL contexts. In L. Hamp-Lyons (Ed.), Aassessing second language writing in academic contexts (pp.241–66). Norwood, NJ : Ablex.
     [Google Scholar]
  10. (1995) : Rating non-native writing : The trouble with holistic scoring. TESOL Quarterly29, 4758–762.
     [Google Scholar]
  11. HAMP-LYONS, L. & B. KROLL.
    (1996) : Issues in ESL writing assessment : An overview. College ESL6, 1:52–72.
     [Google Scholar]
  12. HUNT, K.
    (1965) : Grammatical structures written at three grade levels. Champaign, IL : National Council of Teachers of English.
     [Google Scholar]
  13. KROLL, B.
    (1998) : Assessing Writing Abilities. Annual Review of Applied Linguistics, 18: 219–240
     [Google Scholar]
  14. LAUFER, B. & NATION, P.
    (1995) : Vocabulary size and use : Lexical richness in L2 written production. Applied Linguistics, 16, 4:307–321.
     [Google Scholar]
  15. POLIO, CG.
    (1997) : Measures of linguistic accuracy in second language writing research. Language Learning47, 1:101–143.
     [Google Scholar]
  16. SCOTT, M.
    (1996): Wordsmith Tools. Cambridge University Press.
  17. VAUGHAN, C.
    (1991) : Holistic assessment: What goes on in the raters' minds? InL. Hamp-Lyons (Ed.), assessing second language writing in academic contexts (pp.111–26). Norwood, NJ : Ablex.
     [Google Scholar]
/content/journals/10.1075/itl.127-128.02li
Loading
  • Article Type: Research Article

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error