1887
  1. Home
  2. e-Journals
  3. Chinese as a Second Language. The journal of the Chinese Language Teachers Association, USA
  4. Volume 51, Issue 3
  5. Article
Volume 51, Issue 3
  • ISSN 2451-828x
  • E-ISSN: 2451-8298
USD
Buy:$35.00 + Taxes

Abstract

This study empirically examines the rating criteria used to assess U.S. college students’ CSL (Chinese as a Second Language) oral performance by analyzing teachers’ assessment of these performances at different proficiency levels. The researcher videotaped ten speeches, and three ACTFL-trained raters assessed oral performance in these samples. The researcher then selected three samples (Samples 1, 2, and 3) to represent Novice High, Intermediate High, and Advanced Low levels. The researcher developed 20 rating items through interviewing ten experienced CSL teachers and running an Exploratory Factor Analysis (EFA) on teachers’ assessments of speech samples. After that, 104 CSL teachers used these rating items to assess the aforementioned samples. The EFAs of teachers’ assessments led to three corresponding rating criteria models (Models 1, 2, and 3). Both Models 2 and 3 for Samples 2 and 3, respectively, were five-criterion models, consisting of fluency, conceptual understanding, content richness, communication appropriateness, and communication clarity. Model 1 for Sample 1 was a four-criterion model, in which the items in communication appropriateness and content richness showed high correlations, and therefore were merged into one category; the other three criteria remained the same. Comparisons of the three models demonstrated that the criteria were constant. The ANOVAs showed that the proficiency levels of these oral performances differed significantly across all five rating criteria. This study empirically supports CSL teachers’ use of constant rating criteria to assess different levels of oral performance. It also provides Chinese teachers with rating criteria they can use to assess U.S. college students’ CSL oral performance.

© 2016 John Benjamins
Loading

Article metrics loading...

/content/journals/10.1075/csl.51.3.04che
2017-01-30
2024-10-03

  • From This Site

    /content/journals/10.1075/csl.51.3.04che
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. ACTFL
    1987 ACTFL Chinese Proficiency Guidelines. Hastings-on-Hudson, NY: ACTFL.
     [Google Scholar]
  2. 1999 The ACTFL Proficiency Guidelines—Speaking Revised. Hastings-on-Hudson, NY: ACTFL.
     [Google Scholar]
  3. 2012 The ACTFL Proficiency Guidelines—Speaking. Hastings-on-Hudson, NY: ACTFL.
     [Google Scholar]
  4. Adams, Marianne L
    1980 “Five Co-occurring Factors in Speaking Proficiency.” InMeasuring Spoken Proficiency, edited by James Firth , 1–6. Washington, DC: Georgetown University Press.
     [Google Scholar]
  5. Bachman, Lyle F
    1990 Fundamental Considerations in Language Testing. Oxford: Oxford University Press.
     [Google Scholar]
  6. Canale, Michael
    1983 “From Communicative Competence to Communicative Language Pedagogy.” InLanguage and Communication, edited by Jack C. Richards and Richard W. Schmidt , 2–27. London: Longman.
     [Google Scholar]
  7. Canale, Michael , and Merrill Swain
    1980 “Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing.” Applied Linguistics1: 1–47. doi: 10.1093/applin/1.1.1
     https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  8. Child, Dennis
    1990 The Essentials of Factor Analysis (2nd ed.). London: Cassel Educational Limited.
     [Google Scholar]
  9. Chomsky, Noam
    1965 Aspects of the Theory of Syntax. MIT Press.
     [Google Scholar]
  10. Council of Europe
    2001 Common European Framework of Reference for Languages: Learning, Teaching and Assessment. Cambridge: Cambridge University Press. Retrieved fromwww.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf.
     [Google Scholar]
  11. De Jong, John H.A.L. , and Lieneke W. van Ginkel
    1992 “Dimensions in Oral Foreign Language Proficiency.” InThe Construct of Language Proficiency, edited by Ludo Verhoeven and John H.A.L. De Jong , 187–206. Amsterdam: John Benjamins. doi: 10.1075/z.62.19jon
     https://doi.org/10.1075/z.62.19jon [Google Scholar]
  12. Fulcher, Glenn
    1996 “Does Thick Description Lead to Smart Tests? A Data-based Approach to Rating Scale Construction.” Language Testing13(2): 208–238. doi: 10.1177/026553229601300205
     https://doi.org/10.1177/026553229601300205 [Google Scholar]
  13. 2003 Testing Second Language Speaking. London: Pearson Education.
     [Google Scholar]
  14. 2010 Practical Language Testing. London: Hodder Education.
     [Google Scholar]
  15. Hadden, Betsy L
    1991 “Teacher and Nonteacher Perceptions of Second Language Communication.” Language Learning41: 1–24. doi: 10.1111/j.1467−1770.1991.tb00674.x
     https://doi.org/10.1111/j.1467−1770.1991.tb00674.x [Google Scholar]
  16. Hall, Joan Kelly
    1993 “The Role of Practice in Interaction with Implications for Learning Another Language.” Applied Linguistics14: 145–166. doi: 10.1093/applin/14.2.145
     https://doi.org/10.1093/applin/14.2.145 [Google Scholar]
  17. Higgs, Theodore V. , and Ray Clifford
    1982 “The Push towards Communication.” InCurriculum, Competence, and the Foreign Language Teacher, edited by Theodore V. Higgs , 57–79. Lincolnwood, IL: National Textbook Company.
     [Google Scholar]
  18. Hudson, Thom
    2005 “Trends in Assessment Scales and Criterion-Referenced Language Assessment.” Annual Review of Applied Linguistics25: 205–227. doi: 10.1017/S0267190505000115
     https://doi.org/10.1017/S0267190505000115 [Google Scholar]
  19. Hymes, Dell
    1972 “On Communicative Competence.” InSociolinguistics, edited by John B. Pride and Janet Holmes , 269–93. Harmondsworth, Middleesex: Penguin.
     [Google Scholar]
  20. Iwashita, Noriko , Annie Brown , Tim McNamara , and Sally O’Hagan
    2008 “Assessed Levels of Second Language Speaking Proficiency: How Distinct?” Applied Linguistics29(1): 24–49. doi: 10.1093/applin/amm017
     https://doi.org/10.1093/applin/amm017 [Google Scholar]
  21. Iwashita, Noriko , and Elisabeth Grove
    2003 “A Comparison of Analytic and Holistic Scales in the Context of a Specific-Purpose Speaking Test.” Prospect18(3): 25–35.
     [Google Scholar]
  22. Jackendoff, Ray
    2002 Foundations of Language. Oxford, New York: Oxford University Press. doi: 10.1093/acprof:oso/9780198270126.001.0001
     https://doi.org/10.1093/acprof:oso/9780198270126.001.0001 [Google Scholar]
  23. Jin, Tan , and Barley Mak
    2013 “Distinguishing Features in Scoring a Chinese Speaking Performance: How Do They Work?” Language Testing30(1): 23–47. doi: 10.1177/0265532212442637
     https://doi.org/10.1177/0265532212442637 [Google Scholar]
  24. Lennon, Paul
    1990 “Investigating Fluency in EFL: A Quantitative Approach.” Language Learning3: 387–417. doi: 10.1111/j.1467−1770.1990.tb00669.x
     https://doi.org/10.1111/j.1467−1770.1990.tb00669.x [Google Scholar]
  25. Ministry of Education and State Language Commission, the People’s Republic of China
    2011 Spoken Chinese Proficiency Grading Standards and Testing Guidelines. Beijing: Beijing Language and Culture University Press.
     [Google Scholar]
  26. North, Brian
    2000 Scale for Rating Language Proficiency: Descriptive Models, Formulation Styles, and Presentation Formats (TOEFL Research Paper). Princeton, NJ: Educational Testing Service.
     [Google Scholar]
  27. Plough, India C. , Sarah L. Briggs , and Sarah Van Bonn
    2010 “A Multi-method Analysis of Evaluation Criteria Used to Assess the Speaking Proficiency of Graduate Student Instructors.” Language Testing27(2): 235–260. doi: 10.1177/0265532209349469
     https://doi.org/10.1177/0265532209349469 [Google Scholar]
  28. Sato, Takanon
    2012 “The Contribution of Test-Takers’ Speech Content to Scores on an English Oral Proficiency Test.” Language Testing29(2): 223–241. doi: 10.1177/0265532211421162
     https://doi.org/10.1177/0265532211421162 [Google Scholar]
  29. Sawaki, Yasuyo
    2007 “Construct Validation of Analytic Rating Scales in a Speaking Assessment: Reporting a Score Profile and a Composite.” Language Testing24(3): 355–390. doi: 10.1177/0265532207077205
     https://doi.org/10.1177/0265532207077205 [Google Scholar]
  30. Schumann, John H
    1976 “Social Distance as a Factor in Second Language Acquisition.” Language Learning26: 135–143. doi: 10.1111/j.1467−1770.1976.tb00265.x
     https://doi.org/10.1111/j.1467−1770.1976.tb00265.x [Google Scholar]
  31. Suhr, Diana
    2004 Reliability, Exploratory and Confirmatory Factor Analysis for the Scale of Athletic Priorities. White paper. University of Northern Colorado.
     [Google Scholar]
  32. Tabachnick, Babara G. , and Linda S. Fidell
    2007 Using Multivariate Statistics (5th ed.). Boston, MA: Allyn and Bacon.
     [Google Scholar]
  33. The Office of Chinese Language Council International
    2007 Chinese Language Proficiency Scales for Speakers of Other Languages. Beijing: Foreign Language Teaching and Research Press.
     [Google Scholar]
  34. 2008 International Curriculum for Chinese Language Education. Beijing: Foreign Language Teaching and Research Press.
     [Google Scholar]
  35. Walker, Galal
    2000 “Performed Culture: Learning to Participate in Another Culture.” InLanguage Policy and Pedagogy, edited by Richard Lambert and Elana Shohamy , 221–236. Amsterdam: John Benjamins. doi: 10.1075/z.96.14wal
     https://doi.org/10.1075/z.96.14wal [Google Scholar]
  36. Wang, Jimin
    2002 “A Study of the Scoring of Three Types of Oral Test Items (in Chinese).” Chinese Teaching in the World4: 63–77.
     [Google Scholar]
  37. Young, Richard F
    1999 “Sociolinguistic Approach to SLA.” Annual Review of Applied Linguistics19: 105–131. doi: 10.1017/S0267190599190068
     https://doi.org/10.1017/S0267190599190068 [Google Scholar]
/content/journals/10.1075/csl.51.3.04che
Loading
  • Article Type: Research Article
Keyword(s): Chinese as a second language; exploratory factor analysis; language assessment; oral performance; rating criteria

Most Cited

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error