Abstract
This paper discusses the development of an assessment to satisfy the International Civil Aviation Organization (ICAO) Language Proficiency Requirements. The Versant Aviation English Test utilizes speech recognition technology and a computerized testing platform, such that test administration and scoring are fully automated. Developed in collaboration with the U.S. Federal Aviation Administration, this 25-minute test is delivered via a telephone or computer. Two issues of interest are discussed. The first concerns the practicalities of assessing candidates in each of six separate dimensions of spoken proficiency: Pronunciation, Structure, Vocabulary, Fluency, Comprehension, and Interactions. Although an automated scoring system can objectively segregate these skills, we question whether human raters have the capacity to do this in oral interviews. The second issue discussed is how an automated test can provide a valid assessment of spoken interactions. Tasks were designed to simulate the information exchange between pilots and controllers on which candidates’ proficiency in ‘Interactions’ could be measured, for example, by eliciting functions such as correcting miscommunications and providing clarification. It is argued that candidate ability can be probed and estimated in a fair and standardized way by presenting a series of independent items which are targeted in difficulty at the various ICAO levels.
© 2009 John Benjamins Publishing Company
Article metrics loading...
/content/journals/10.2104/aral0927
2009-01-01
2024-03-28
-
/content/journals/10.2104/aral0927
dcterms_title,dcterms_subject,pub_keyword
-contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
10
5
Full text loading...
References
-
Alderson, J. Charles
(2008) Final report on a survey of aviation English tests. Unpublished report.
[Google Scholar]
-
Bachman, Lyle F ; Palmer, Adrian S
https://doi.org/10.1111/j.1467-1770.1981.tb01373.x
[Google Scholar]
-
Bachman, Lyle F. ; Palmer, Adrian S.
(1996) Language Testing in Practice. Oxford. Oxford University Press.
[Google Scholar]
-
Brown, Annie
(
2003) Interviewer variation and the co–construction of speaking proficiency.
Language Testing, 20(1), 1–25. doi:
10.1191/0265532203lt242oa
https://doi.org/10.1191/0265532203lt242oa
[Google Scholar]
-
Brown, Gillian ; Yule, George
https://doi.org/10.1017/CBO9780511805226
[Google Scholar]
-
Brown, Gillian ; Anderson, Anne ; Shillcock, Richard ; Yule, George
(1984) Teaching Talk: Strategies for Production and Assessment.,Cambridge: Cambridge University Press.
[Google Scholar]
-
Fulcher, Glenn
https://doi.org/10.1177/026553229601300103
[Google Scholar]
-
Henning, Grant
(1987) A Guide to Language Testing. Boston, MA: Heinle and Heinle.
[Google Scholar]
-
Hinofotis, Frances B.
(1983) The structure of oral communication is an educational environment: a comparison of factor–analytic rotational procedures. In Oller, J.W., Jr (Ed.), Issues in Language Testing Research (pp.170–187). Rowley, MA: Newbury House.
[Google Scholar]
-
International Civil Aviation Organization (ICAO)
International Civil Aviation Organization (ICAO) (2004) Manual on the Implementing of ICAO Language Proficiency Requirements. Montreal: ICAO.
[Google Scholar]
-
Johnson, Marysia
(2001) The Art of Non–conversation: A Reexamination of the Validity of the Oral Proficiency Interview. New Haven, CT: Yale University Press.
[Google Scholar]
-
Lantolf, James ; Frawley, William
https://doi.org/10.1111/j.1540-4781.1985.tb04801.x
[Google Scholar]
-
Linde, Charlotte
(
1988) The quantitative study of communicative success: Politeness and accidents in aviation discourse.
Language in Society, 17(3), 375–399 doi:
10.1017/S0047404500012951
https://doi.org/10.1017/S0047404500012951
[Google Scholar]
-
McNamara, Tim
(1996) Measuring Second Language Performance. London: Longman.
[Google Scholar]
-
Morrow, Daniel G ; Rodvold, Michelle ; Lee, Alfred
(
1994) Nonroutine transactions in controller–pilot communication.
Discourse Processes, 17(2), 235–258. doi:
10.1080/01638539409544868
https://doi.org/10.1080/01638539409544868
[Google Scholar]
-
Orr, Michael
https://doi.org/10.1016/S0346-251X(02)00002-7
[Google Scholar]
-
Pearson
Pearson 2008 Versant Aviation English Test: Test description and validation summary. Palo Alto, CA: Author.
[Google Scholar]
-
Qian, David
(
2009) Comparing direct and semi-direct modes for speaking assessment: Affective effects on test-takers.
Language Assessment Quarterly, 6(2), 113–125. doi:
10.1080/15434300902800059
https://doi.org/10.1080/15434300902800059
[Google Scholar]
-
Ross, Steven
https://doi.org/10.1177/026553229200900205
[Google Scholar]
-
Sacks, Harvey ; Schegloff, Emanuel ; Jefferson, Gail
(
1974) A simplest systematics for the organization of turn-taking for conversation.
Language, 50(4), 696–735. doi:
10.1353/lan.1974.0010
https://doi.org/10.1353/lan.1974.0010
[Google Scholar]
-
Schegloff, Emanuel ; Jefferson, Gail ; Sacks, Harvey
(
1977) The preference for self-correction in the organization of repair in conversation.
Language, 53(2), 361–382. doi:
10.1353/lan.1977.0041
https://doi.org/10.1353/lan.1977.0041
[Google Scholar]
-
Silverman, David
(1976) Interview talk: Bringing off a research instrument. In Silverman, D. and Jones, J. (Eds.), Organisational Work: The language of grading, the grading of language (pp.133–150). London: Collier Macmillan.
[Google Scholar]
-
Swain, Merrill
(2001) Examining dialogue: Another approach to content specification and to validating inferences drawn from test scores. Language Testing, 18(3), 275–302.
[Google Scholar]
-
Tannen, Deborah
(1984) Conversational Style: Analyzing Talk among Friends. Norwood, NJ: Ablex.
[Google Scholar]
-
van Lier, L
(
1989) Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation.
TESOL Quarterly, 23(3), 489–508. doi:
10.2307/3586922
https://doi.org/10.2307/3586922
[Google Scholar]
-
Weir, Cyril J.
(
2005)
Language Testing and Validation: An Evidence-based Approach. New York: Palgrave Macmillan. doi:
10.1057/9780230514577
https://doi.org/10.1057/9780230514577
[Google Scholar]
http://instance.metastore.ingenta.com/content/journals/10.2104/aral0927