1887
  1. Home
  2. e-Journals
  3. English Text Construction
  4. Volume 13, Issue 1
  5. Article
Volume 13, Issue 1
  • ISSN 1874-8767
  • E-ISSN: 1874-8775
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This study aims to determine the linguistic and discoursal differences in essays produced by Iranian test-takers of TOEFL-iBT in response to integrated and independent writing tasks. A sample of 40 essays, written by 20 Iranian test-takers of scored integrated and independent writing tasks, was compared and analyzed in terms of the four latent constructs of text easability (fourteen variables), cohesion (nine variables), lexical sophistication (nineteen variables), and syntactic complexity (six variables), using the Coh-Metrix 3.0 program. Results indicate differences in the linguistic and discoursal features of integrated and independent writing tasks. The findings reveal that the scores on writing tasks of EFL test-takers can be anchored empirically through the analysis of some discourse qualities like cohesion. Independent tasks contain more connectives and particles so they can result in better discourse structure organization and the generation of more cohesive devices. Stakeholders of the test should verify test constructs in terms of particular contexts like EFL and communicative views of language proficiency. Consequently, the findings contribute to the ongoing validity argument on TOEFL-iBT writing tasks for designing and interpreting scoring schemes for the writing component of the test.

© John Benjamins Publishing Company
Loading

Article metrics loading...

/content/journals/10.1075/etc.00036.sho
2020-07-24
2024-12-12

  • From This Site

    /content/journals/10.1075/etc.00036.sho
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Banerjee, Jayanti, Florencia Franceschina & Anne M. Smith
    2007 Documenting features of written language production typical of different IELTS band score levels (IELTS Research Reports). Canberra, Australia: IELTS Australia Pty and British Council.
     [Google Scholar]
  2. Biber, Douglas & Bethany Gray
    2013 Discourse characteristics of writing and speaking task types on the TOEFL-iBT test: A lexico-grammatical analysis. TOEFL-iBT research report. Retrieved from www.ets.org/Media/Research/pdf/RR-13-04.pdf (6May 2020).
     [Google Scholar]
  3. Chapelle, Carol. A., Mary K. Enright & Joan M. Jamieson
    2008 Building a Validity Argument for the Test of English as a Foreign Language TM. New York, NY: Routledge.
     [Google Scholar]
  4. Cho, Yeonsuk, Frank Rijmen & Jakub Novak
    2013 Investigating the effects of prompt characteristics on the comparability of TOEFL-iBT integrated writing tasks. Language Testing30 (4): 513–534. doi:  10.1177/0265532213478796
     https://doi.org/10.1177/0265532213478796 [Google Scholar]
  5. Connor, Ulla
    1990 Linguistic/rhetorical measures of international persuasive student writing. Research in the Teaching of English24 (1): 67–87.
     [Google Scholar]
  6. Crossley, Scott A., Amanda M. Clevinger & Youjin Kim
    2014 The role of lexical properties and cohesive devices in text integration and their effect on human ratings of speaking proficiency. Language Assessment Quarterly11(3): 250–270. doi:  10.1080/15434303.2014.926905
     https://doi.org/10.1080/15434303.2014.926905 [Google Scholar]
  7. Crossley, Scott A., & Danielle S. McNamara
    2012 Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading35(2): 115–135. 10.1111/j.1467‑9817.2010.01449.x
     https://doi.org/10.1111/j.1467-9817.2010.01449.x [Google Scholar]
  8. Crossley, Scott A. & Danielle S. McNamara
    2013 Applications of text analysis tools for spoken response grading. Language Learning & Technology17(2): 171–192.
     [Google Scholar]
  9. Cumming, Alister, Robert Kantor, Kyoko Baba, Keanre Eouanzoui, Usman Erdoosy & Mark James
    2006 Analysis of discourse features and verification of scoring levels for independent and integrated tasks for the new TOEFL. Princeton, NJ: Educational Testing Service. doi:  10.1002/j.2333‑8504.2005.tb01990.x
     https://doi.org/10.1002/j.2333-8504.2005.tb01990.x [Google Scholar]
  10. Cumming, Alister, Robert Kantor, Kyoko Baba, Usman Erdoosy, Keanre Eouanzoui & Mark James
    2005 Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing10(1): 5–43. 10.1016/j.asw.2005.02.001
     https://doi.org/10.1016/j.asw.2005.02.001 [Google Scholar]
  11. Cumming, Alister, Robert Kantor, Donald E. Powers, Terry Santos & Carol Taylor
    2000 TOEFL 2000 writing framework: A working paper. InTOEFL-MS-18. Princeton, NJ: Educational Testing Service.
     [Google Scholar]
  12. Deane, Paul
    2013 On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing18(1): 7–24. 10.1016/j.asw.2012.10.002
     https://doi.org/10.1016/j.asw.2012.10.002 [Google Scholar]
  13. Educational Testing Service
    Educational Testing Service 2018 Reliability and Comparability of TOEFL-iBT Scores. Princeton, NJ: Educational Testing Service.
     [Google Scholar]
  14. Engber, Cheryl. A.
    1995 The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing4(2): 139–155. 10.1016/1060‑3743(95)90004‑7
     https://doi.org/10.1016/1060-3743(95)90004-7 [Google Scholar]
  15. Ferris, Dana R.
    1994 Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly28(2): 414–420. 10.2307/3587446
     https://doi.org/10.2307/3587446 [Google Scholar]
  16. Frase, Lawrence T., Joseph Faletti, April Ginther & Leslie Grant
    1999 Computer Analysis of the TOEFL Test of Written English [TOEFL Research Report No. 64]. Princeton, NJ: ETS.
     [Google Scholar]
  17. Gebril, Atta & Lia Plakans
    2009 Investigating source use, discourse features, and process in integrated writing tests. Spaan Fellow Working Papers in Second/foreign Language Assessment7: 47–84.
     [Google Scholar]
  18. Graesser, Arthur C. & Danielle S. McNamara
    2011 Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science3 (2): 371–398. doi:  10.1111/j.1756‑8765.2010.01081.x
     https://doi.org/10.1111/j.1756-8765.2010.01081.x [Google Scholar]
  19. Graesser, Arthur C., Danielle S. McNamara, Max M. Louwerse & Zhiqiang Cai
    2004 Coh-Metrix: Analysis of text on cohesion and language. Behavioral Research Methods, Instruments, and Computers36(2): 193–202. 10.3758/BF03195564
     https://doi.org/10.3758/BF03195564 [Google Scholar]
  20. Graesser, Arthur C., Murray Singer & Tom Trabasso
    1994 Constructing inferences during narrative text comprehension. Psychological Review101(3): 371–395. 10.1037/0033‑295X.101.3.371
     https://doi.org/10.1037/0033-295X.101.3.371 [Google Scholar]
  21. Grant, Leslie & April Ginther
    2000 Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing9(2): 123–145. 10.1016/S1060‑3743(00)00019‑9
     https://doi.org/10.1016/S1060-3743(00)00019-9 [Google Scholar]
  22. Guo, Liang
    2011 Product and process in TOEFL-iBT independent and integrated writing tasks: An investigation of construct validity. Unpublished doctoral dissertation, Georgia State University.
  23. Guo, Liang, Scott A. Crossley & Danielle S. McNamara
    2013 Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing18(3): 218–238. doi:  10.1016/j.asw.2013.05.002
     https://doi.org/10.1016/j.asw.2013.05.002 [Google Scholar]
  24. Hamp-Lyons, Liz & Barbara Kroll
    1996 Issues in ESL writing assessment: An overview. College ESL6(1): 52–72.
     [Google Scholar]
  25. Jarvis, Scott
    2002 Short texts, best-fitting curves and new measures of lexical diversity. Language Testing19(1): 57–84. 10.1191/0265532202lt220oa
     https://doi.org/10.1191/0265532202lt220oa [Google Scholar]
  26. Jin, Wenjun
    2001 A quantitative study of cohesion in Chinese graduate students’ writing: Variations across genres and proficiency levels (ERIC Document Reproduction Service No. ED 452 726).
     [Google Scholar]
  27. Kintsch, Walter
    1998 Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press.
     [Google Scholar]
  28. Knoch, Ute, Susy Macqueen & Sally O’Hagan
    2014 An Investigation of the Effect of Task Type on the Discourse Produced by Students at Various Score Levels in the TOEFL-iBT® Writing Test [Research Report No. RR-14–43]. Princeton, NJ: Educational Testing Service. doi:  10.1002/ets2.12038
     https://doi.org/10.1002/ets2.12038 [Google Scholar]
  29. Knoch, Ute
    2009 Diagnostic Assessment of Writing: The Development and Validation of a Rating Scale. Frankfurt: Peter Lang.
     [Google Scholar]
  30. Kyle, Kristopher & Scott A. Crossley
    2016 The relationship between lexical sophistication and independent and source-based writing. Journal of Second Language Writing34: 12–24. doi:  10.1016/j.jslw.2016.10.003
     https://doi.org/10.1016/j.jslw.2016.10.003 [Google Scholar]
  31. Landauer, Thomas K., Danielle S. McNamara, Simon Dennis & Walter Kintsch
    (eds) 2007 LSA: A Road to Meaning. Mahwah, NJ: Lawrence Erlbaum Associates.
     [Google Scholar]
  32. Leki, Ilona & Joan Carson
    1997 Completely different worlds: EAP and the writing experiences of ESL students in university courses. TESOL Quarterly31(1): 36–69. 10.2307/3587974
     https://doi.org/10.2307/3587974 [Google Scholar]
  33. McNamara, Danielle S., Scott A. Crossley & Philip M. McCarthy
    2010 Linguistic features of writing quality. Written Communication27(1): 57–86. 10.1177/0741088309351547
     https://doi.org/10.1177/0741088309351547 [Google Scholar]
  34. McNamara, Danielle S. & Arthur C. Graesser
    2012 Coh-Metrix: An automated tool for theoretical and applied natural language processing. InApplied Natural Language Processing: Identification, Investigation, and Resolution, P. M. McCarthy, & C. Boonthum-Denecke (eds), 188–205. Hershey, PA: IGI Global. 10.4018/978‑1‑60960‑741‑8.ch011
     https://doi.org/10.4018/978-1-60960-741-8.ch011 [Google Scholar]
  35. McNamara, Danielle S. & Joe Magliano
    2009 Toward a comprehensive model of comprehension. Psychology of Learning and Motivation – Advances in Research and Theory51: 297–384. doi:  10.1016/S0079‑7421(09)51009‑2
     https://doi.org/10.1016/S0079-7421(09)51009-2 [Google Scholar]
  36. McNamara, Danielle S., Max M. Louwerse, Philip M. McCarthy & Arthur C. Graesser
    2010 Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes47(4): 292–330. 10.1080/01638530902959943
     https://doi.org/10.1080/01638530902959943 [Google Scholar]
  37. Norris, John M. & Lourdes Ortega
    2009 Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics30(4): 555–578. 10.1093/applin/amp044
     https://doi.org/10.1093/applin/amp044 [Google Scholar]
  38. Plakans, Lia, Atta Gebril & Zeynep Bilki
    2019 Shaping a score: The impact of fluency, accuracy, and complexity on integrated skills performances. Journal of Language Testing36 (2): 161–179. doi:  10.1177/0265532216669537
     https://doi.org/10.1177/0265532216669537 [Google Scholar]
  39. Riazi, Mehdi A.
    2016 Comparing writing performance in TOEFL-iBT and academic assignments: An exploration of textual features. Assessing Writing28, 15–27. doi:  10.1016/j.asw.2016.02.001
     https://doi.org/10.1016/j.asw.2016.02.001 [Google Scholar]
  40. Tierney, Robert J. & Timothy Shanahan
    2001 Research on the reading-writing relationship: Interactions, transactions, and outcomes. InHandbook of Reading Research, R. Barr, M. L. Kamil, P. Mosenthal, & P. D. Pearson (eds), 246–280. New York: Longman.
     [Google Scholar]
  41. Weigle, Sara C.
    2004 Integrating reading and writing in a competency test for non-native speakers of English. Assessing Writing9(1): 27–55. 10.1016/j.asw.2004.01.002
     https://doi.org/10.1016/j.asw.2004.01.002 [Google Scholar]
/content/journals/10.1075/etc.00036.sho
Loading
Assessing writing performance in TOEFL-iBT
English Text Construction 13, 84 (2020); https://doi.org/10.1075/etc.00036.sho
/content/journals/10.1075/etc.00036.sho
/content/journals/10.1075/etc.00036.sho
Loading

Data & Media loading...

  • Article Type: Research Article
Keyword(s): Coh-Metrix; independent writing task; integrated writing task; TOEFL-iBT; writing assessment

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error