Volume 51, Issue 3
  • ISSN 2451-828x
  • E-ISSN: 2451-8298
Buy:$35.00 + Taxes


This study empirically examines the rating criteria used to assess U.S. college students’ CSL (Chinese as a Second Language) oral performance by analyzing teachers’ assessment of these performances at different proficiency levels. The researcher videotaped ten speeches, and three ACTFL-trained raters assessed oral performance in these samples. The researcher then selected three samples (Samples 1, 2, and 3) to represent Novice High, Intermediate High, and Advanced Low levels. The researcher developed 20 rating items through interviewing ten experienced CSL teachers and running an Exploratory Factor Analysis (EFA) on teachers’ assessments of speech samples. After that, 104 CSL teachers used these rating items to assess the aforementioned samples. The EFAs of teachers’ assessments led to three corresponding rating criteria models (Models 1, 2, and 3). Both Models 2 and 3 for Samples 2 and 3, respectively, were five-criterion models, consisting of fluency, conceptual understanding, content richness, communication appropriateness, and communication clarity. Model 1 for Sample 1 was a four-criterion model, in which the items in communication appropriateness and content richness showed high correlations, and therefore were merged into one category; the other three criteria remained the same. Comparisons of the three models demonstrated that the criteria were constant. The ANOVAs showed that the proficiency levels of these oral performances differed significantly across all five rating criteria. This study empirically supports CSL teachers’ use of constant rating criteria to assess different levels of oral performance. It also provides Chinese teachers with rating criteria they can use to assess U.S. college students’ CSL oral performance.


Article metrics loading...

Loading full text...

Full text loading...


  1. ACTFL
    1987ACTFL Chinese Proficiency Guidelines. Hastings-on-Hudson, NY: ACTFL.
    [Google Scholar]
  2. 1999The ACTFL Proficiency Guidelines—Speaking Revised. Hastings-on-Hudson, NY: ACTFL.
    [Google Scholar]
  3. 2012The ACTFL Proficiency Guidelines—Speaking. Hastings-on-Hudson, NY: ACTFL.
    [Google Scholar]
  4. Adams, Marianne L
    1980 “Five Co-occurring Factors in Speaking Proficiency.” InMeasuring Spoken Proficiency, edited by James Firth , 1–6. Washington, DC: Georgetown University Press.
    [Google Scholar]
  5. Bachman, Lyle F
    1990Fundamental Considerations in Language Testing. Oxford: Oxford University Press.
    [Google Scholar]
  6. Canale, Michael
    1983 “From Communicative Competence to Communicative Language Pedagogy.” InLanguage and Communication, edited by Jack C. Richards and Richard W. Schmidt , 2–27. London: Longman.
    [Google Scholar]
  7. Canale, Michael , and Merrill Swain
    1980 “Theoretical Bases of Communicative Approaches to Second Language Teaching and Testing.” Applied Linguistics1: 1–47. doi: 10.1093/applin/1.1.1
    https://doi.org/10.1093/applin/1.1.1 [Google Scholar]
  8. Child, Dennis
    1990The Essentials of Factor Analysis (2nd ed.). London: Cassel Educational Limited.
    [Google Scholar]
  9. Chomsky, Noam
    1965Aspects of the Theory of Syntax. MIT Press.
    [Google Scholar]
  10. Council of Europe
    2001Common European Framework of Reference for Languages: Learning, Teaching and Assessment. Cambridge: Cambridge University Press. Retrieved fromwww.coe.int/t/dg4/linguistic/Source/Framework_EN.pdf.
    [Google Scholar]
  11. De Jong, John H.A.L. , and Lieneke W. van Ginkel
    1992 “Dimensions in Oral Foreign Language Proficiency.” InThe Construct of Language Proficiency, edited by Ludo Verhoeven and John H.A.L. De Jong , 187–206. Amsterdam: John Benjamins. doi: 10.1075/z.62.19jon
    https://doi.org/10.1075/z.62.19jon [Google Scholar]
  12. Fulcher, Glenn
    1996 “Does Thick Description Lead to Smart Tests? A Data-based Approach to Rating Scale Construction.” Language Testing13(2): 208–238. doi: 10.1177/026553229601300205
    https://doi.org/10.1177/026553229601300205 [Google Scholar]
  13. 2003Testing Second Language Speaking. London: Pearson Education.
    [Google Scholar]
  14. 2010Practical Language Testing. London: Hodder Education.
    [Google Scholar]
  15. Hadden, Betsy L
    1991 “Teacher and Nonteacher Perceptions of Second Language Communication.” Language Learning41: 1–24. doi: 10.1111/j.1467−1770.1991.tb00674.x
    https://doi.org/10.1111/j.1467−1770.1991.tb00674.x [Google Scholar]
  16. Hall, Joan Kelly
    1993 “The Role of Practice in Interaction with Implications for Learning Another Language.” Applied Linguistics14: 145–166. doi: 10.1093/applin/14.2.145
    https://doi.org/10.1093/applin/14.2.145 [Google Scholar]
  17. Higgs, Theodore V. , and Ray Clifford
    1982 “The Push towards Communication.” InCurriculum, Competence, and the Foreign Language Teacher, edited by Theodore V. Higgs , 57–79. Lincolnwood, IL: National Textbook Company.
    [Google Scholar]
  18. Hudson, Thom
    2005 “Trends in Assessment Scales and Criterion-Referenced Language Assessment.” Annual Review of Applied Linguistics25: 205–227. doi: 10.1017/S0267190505000115
    https://doi.org/10.1017/S0267190505000115 [Google Scholar]
  19. Hymes, Dell
    1972 “On Communicative Competence.” InSociolinguistics, edited by John B. Pride and Janet Holmes , 269–93. Harmondsworth, Middleesex: Penguin.
    [Google Scholar]
  20. Iwashita, Noriko , Annie Brown , Tim McNamara , and Sally O’Hagan
    2008 “Assessed Levels of Second Language Speaking Proficiency: How Distinct?” Applied Linguistics29(1): 24–49. doi: 10.1093/applin/amm017
    https://doi.org/10.1093/applin/amm017 [Google Scholar]
  21. Iwashita, Noriko , and Elisabeth Grove
    2003 “A Comparison of Analytic and Holistic Scales in the Context of a Specific-Purpose Speaking Test.” Prospect18(3): 25–35.
    [Google Scholar]
  22. Jackendoff, Ray
    2002Foundations of Language. Oxford, New York: Oxford University Press. doi: 10.1093/acprof:oso/9780198270126.001.0001
    https://doi.org/10.1093/acprof:oso/9780198270126.001.0001 [Google Scholar]
  23. Jin, Tan , and Barley Mak
    2013 “Distinguishing Features in Scoring a Chinese Speaking Performance: How Do They Work?” Language Testing30(1): 23–47. doi: 10.1177/0265532212442637
    https://doi.org/10.1177/0265532212442637 [Google Scholar]
  24. Lennon, Paul
    1990 “Investigating Fluency in EFL: A Quantitative Approach.” Language Learning3: 387–417. doi: 10.1111/j.1467−1770.1990.tb00669.x
    https://doi.org/10.1111/j.1467−1770.1990.tb00669.x [Google Scholar]
  25. Ministry of Education and State Language Commission, the People’s Republic of China
    2011Spoken Chinese Proficiency Grading Standards and Testing Guidelines. Beijing: Beijing Language and Culture University Press.
    [Google Scholar]
  26. North, Brian
    2000Scale for Rating Language Proficiency: Descriptive Models, Formulation Styles, and Presentation Formats (TOEFL Research Paper). Princeton, NJ: Educational Testing Service.
    [Google Scholar]
  27. Plough, India C. , Sarah L. Briggs , and Sarah Van Bonn
    2010 “A Multi-method Analysis of Evaluation Criteria Used to Assess the Speaking Proficiency of Graduate Student Instructors.” Language Testing27(2): 235–260. doi: 10.1177/0265532209349469
    https://doi.org/10.1177/0265532209349469 [Google Scholar]
  28. Sato, Takanon
    2012 “The Contribution of Test-Takers’ Speech Content to Scores on an English Oral Proficiency Test.” Language Testing29(2): 223–241. doi: 10.1177/0265532211421162
    https://doi.org/10.1177/0265532211421162 [Google Scholar]
  29. Sawaki, Yasuyo
    2007 “Construct Validation of Analytic Rating Scales in a Speaking Assessment: Reporting a Score Profile and a Composite.” Language Testing24(3): 355–390. doi: 10.1177/0265532207077205
    https://doi.org/10.1177/0265532207077205 [Google Scholar]
  30. Schumann, John H
    1976 “Social Distance as a Factor in Second Language Acquisition.” Language Learning26: 135–143. doi: 10.1111/j.1467−1770.1976.tb00265.x
    https://doi.org/10.1111/j.1467−1770.1976.tb00265.x [Google Scholar]
  31. Suhr, Diana
    2004Reliability, Exploratory and Confirmatory Factor Analysis for the Scale of Athletic Priorities. White paper. University of Northern Colorado.
    [Google Scholar]
  32. Tabachnick, Babara G. , and Linda S. Fidell
    2007Using Multivariate Statistics (5th ed.). Boston, MA: Allyn and Bacon.
    [Google Scholar]
  33. The Office of Chinese Language Council International
    2007Chinese Language Proficiency Scales for Speakers of Other Languages. Beijing: Foreign Language Teaching and Research Press.
    [Google Scholar]
  34. 2008International Curriculum for Chinese Language Education. Beijing: Foreign Language Teaching and Research Press.
    [Google Scholar]
  35. Walker, Galal
    2000 “Performed Culture: Learning to Participate in Another Culture.” InLanguage Policy and Pedagogy, edited by Richard Lambert and Elana Shohamy , 221–236. Amsterdam: John Benjamins. doi: 10.1075/z.96.14wal
    https://doi.org/10.1075/z.96.14wal [Google Scholar]
  36. Wang, Jimin
    2002 “A Study of the Scoring of Three Types of Oral Test Items (in Chinese).” Chinese Teaching in the World4: 63–77.
    [Google Scholar]
  37. Young, Richard F
    1999 “Sociolinguistic Approach to SLA.” Annual Review of Applied Linguistics19: 105–131. doi: 10.1017/S0267190599190068
    https://doi.org/10.1017/S0267190599190068 [Google Scholar]
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error