Volume 18, Issue 3
  • ISSN 1932-2798
  • E-ISSN: 1876-2700
Buy:$35.00 + Taxes



The prospect of automated scoring for interpreting fluency has prompted investigations into the predictability of human raters’ perceived fluency based on acoustically measured utterance fluency. Recently, Han, Chen, Fu and Fan (2020) correlated ten utterance fluency measures with raters’ perceived fluency ratings. To verify previous correlational patterns, the present study partially replicated Han et al. (2020). Our analysis shows that most of the correlations observed in Han et al. (2020) were successfully replicated. To produce overall interim estimates of the true relationships, we conducted a mini meta-analysis of correlation coefficients reported in six relevant studies, informed by the “continuously cumulating meta-analysis” approach (Braver et al. 2014). We found that phonation time ratio, mean length of run, and speech rate had relatively strong correlations with perceived fluency. We discuss these findings in light of automated fluency assessment and the need for replication and meta-analysis in translation and interpreting studies.


Article metrics loading...

Loading full text...

Full text loading...


  1. Allen, Mike and Raymond Preiss
    1993 “Replication and meta-analysis: A necessary connection.” Journal of Social Behavior & Personality8(6): 9–20.
    [Google Scholar]
  2. Asendorpf, Jens B., et al
    2013 “Recommendations for increasing replicability in psychology.” European Journal of Personality27(2): 108–119. 10.1002/per.1919
    https://doi.org/10.1002/per.1919 [Google Scholar]
  3. Borenstein, Michael, Larry V. Hedges, Julian Higgins, and Hannah R. Rothstein
    2009Introduction to meta-analysis. Chichester: Wiley. 10.1002/9780470743386
    https://doi.org/10.1002/9780470743386 [Google Scholar]
  4. Brandt, Mark J.
    2014 “The Replication Recipe: What makes for a convincing replication?” Journal of Experimental Social Psychology501: 217–224. 10.1016/j.jesp.2013.10.005
    https://doi.org/10.1016/j.jesp.2013.10.005 [Google Scholar]
  5. Braver, Sanford L., Felix J. Thoemmes, and Robert Rosenthal
    2014 “Continuously cumulating meta-analysis and replicability.” Perspectives on Psychological Science9(3): 333–342. 10.1177/1745691614529796
    https://doi.org/10.1177/1745691614529796 [Google Scholar]
  6. Cecot, Michela
    2001 “Pauses in simultaneous interpretation: A contrastive analysis of professional interpreters’ performances.” The Interpreters’ Newsletter111: 63–85.
    [Google Scholar]
  7. Collard, Camille and Bart Defrancq
    2020 Disfluencies in simultaneous interpreting a corpus-based study with special reference to sex. InNew empirical perspectives on translation and interpreting, ed. byLore Vandevoorde, Joke Daems and Bart Defrancq, 264–299. New York: Routledge. 10.4324/9780429030376
    https://doi.org/10.4324/9780429030376 [Google Scholar]
  8. Corey, David M., William P. Dunlap, and Michael J. Burke
    1998 “Averaging correlations: Expected values and bias in combined Pearson rs and Fisher’s z transformations.” Journal of General Psychology125(3): 245–262. 10.1080/00221309809595548
    https://doi.org/10.1080/00221309809595548 [Google Scholar]
  9. Crandall, Christian S. and Jeffrey W. Sherman
    2016 “On the scientific superiority of conceptual replications for scientific progress.” Journal of Experimental Social Psychology661: 93–99. 10.1016/j.jesp.2015.10.002
    https://doi.org/10.1016/j.jesp.2015.10.002 [Google Scholar]
  10. Easley, Richard W., Charles S. Madden, and Mark G. Dunn
    2000 “Conducting marketing science: The role of replication in the research process.” Journal of Business Research48(1): 83–92. 10.1016/S0148‑2963(98)00079‑4
    https://doi.org/10.1016/S0148-2963(98)00079-4 [Google Scholar]
  11. Gile, Daniel
    2009 “Interpreting studies: a critical view from within.” MonTI11: 135–155. 10.6035/MonTI.2009.1.6
    https://doi.org/10.6035/MonTI.2009.1.6 [Google Scholar]
  12. Goh, Jin X., Judith A. Hall, and Robert Rosenthal
    2016 “Mini meta-analysis of your own studies: Some arguments on why and a primer on how.” Social and Personality Psychology Compass10(10): 535–549. 10.1111/spc3.12267
    https://doi.org/10.1111/spc3.12267 [Google Scholar]
  13. Han, Chao and Ke-rui An
    2021 “Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s).” Perspectives29(6): 917–933. 10.1080/0907676X.2020.1852293
    https://doi.org/10.1080/0907676X.2020.1852293 [Google Scholar]
  14. Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan
    2020 “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting22(2): 211–237. 10.1075/intp.00040.han
    https://doi.org/10.1075/intp.00040.han [Google Scholar]
  15. Han, Chao
    2015 “(Para)linguistic correlates of perceived fluency in English-to-Chinese simultaneous interpretation.” International Journal of Comparative Literature and Translation Studies3(4): 32–37. 10.7575/aiac.ijclts.v.3n.4p.32
    https://doi.org/10.7575/aiac.ijclts.v.3n.4p.32 [Google Scholar]
  16. Hoeppner, Sven
    2019 “A note on replication analysis.” International Review of Law and Economics591: 98–102. 10.1016/j.irle.2019.05.004
    https://doi.org/10.1016/j.irle.2019.05.004 [Google Scholar]
  17. Hubbard, Raymond and J. Scott Armstrong
    1994 “Replications and extensions in marketing: Rarely published but quite contrary.” International Journal of Research in Marketing11(3): 233–248. 10.1016/0167‑8116(94)90003‑5
    https://doi.org/10.1016/0167-8116(94)90003-5 [Google Scholar]
  18. Lee, Jieun
    2008 “Rating scales for interpreting performance assessment.” The Interpreter and Translator Trainer2(2): 165–184. 10.1080/1750399X.2008.10798772
    https://doi.org/10.1080/1750399X.2008.10798772 [Google Scholar]
  19. Mead, Peter
    2000 “Control of pauses by trainee interpreters in their A and B languages.” The Interpreters’ Newsletter101: 89–102.
    [Google Scholar]
  20. 2005 “Methodological issues in the study of interpreters’ fluency.” The Interpreters’ Newsletter131: 39–63.
    [Google Scholar]
  21. Mellinger, Christopher D. and Thomas A. Hanson
    2017Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge. 10.4324/9781315647845
    https://doi.org/10.4324/9781315647845 [Google Scholar]
  22. 2019 “Meta-analyses of simultaneous interpreting and working memory.” Interpreting21(2): 165–195. 10.1075/intp.00026.mel
    https://doi.org/10.1075/intp.00026.mel [Google Scholar]
  23. 2020 “Meta-analysis and replication in interpreting studies.” Interpreting22(1): 140–149. 10.1075/intp.00037.mel
    https://doi.org/10.1075/intp.00037.mel [Google Scholar]
  24. Olalla-Soler, Christian
    2020 “Practices and attitudes toward replication in empirical translation and interpreting studies.” Target32(1): 3–36. 10.1075/target.18159.ola
    https://doi.org/10.1075/target.18159.ola [Google Scholar]
  25. Park, Jung Hwan, Olesya Venger, Doo Yeon Park, and Leonard N. Reid
    2015 “Replication in advertising research, 1980–2012: A longitudinal analysis of leading advertising journals.” Journal of Current Issues and Research in Advertising36(2): 115–135. 10.1080/10641734.2015.1023874
    https://doi.org/10.1080/10641734.2015.1023874 [Google Scholar]
  26. Pöchhacker, Franz
    2011 “Replication in research on quality in conference interpreting.” T&I Review11: 35–58.
    [Google Scholar]
  27. Pöchhacker, Franz and Cornelia Zwischenberger
    2010 “Survey on quality and role: Conference interpreters’ expectations and self-perceptions.” aiic.orghttps://aiic.org/document/9646Last accessed20 January 2022.
    [Google Scholar]
  28. Porte, Graeme and Keith Richards
    2012 “Replication in second language writing research.” Journal of Second Language Writing21 (3): 284–293. 10.1016/j.jslw.2012.05.002
    https://doi.org/10.1016/j.jslw.2012.05.002 [Google Scholar]
  29. Porte, Graeme
    2013 “Who needs replication?” Computer assisted language instruction Consortium Journal30 (1): 10–15. https://www.jstor.org/stable/calicojournal.30.1.10. 10.11139/cj.30.1.10‑15
    https://doi.org/10.11139/cj.30.1.10-15 [Google Scholar]
  30. Rennert, Sylvi
    2010 “The impact of fluency on the subjective assessment of interpreting quality.” The Interpreters’ Newsletter151: 101–115.
    [Google Scholar]
  31. Rosenthal, Robert and Ralph L. Rosnow
    1984 “Applying Hamlet’s question to the ethical conduct of research: A conceptual addendum.” American Psychologist39(5): 561–563. 10.1037/0003‑066X.39.5.561
    https://doi.org/10.1037/0003-066X.39.5.561 [Google Scholar]
  32. Saldanha, Gabriela and Sharon O’Brien
    2013Research methodologies in translation studies. London: Routledge. 10.4324/9781315760100
    https://doi.org/10.4324/9781315760100 [Google Scholar]
  33. Segalowitz, Norman
    2010Cognitive bases of second language fluency. New York: Routledge. 10.4324/9780203851357
    https://doi.org/10.4324/9780203851357 [Google Scholar]
  34. Song, Shuxian
    2020 Fluency in simultaneous interpreting of trainee interpreters: The perspectives of cognitive, utterance and perceived fluency. Unpublished PhD dissertation. The Hong Kong Polytechnic University. https://theses.lib.polyu.edu.hk/handle/200/10418
  35. Stanley, David J. and Jeffrey R. Spence
    2014 “Expectations for replications: Are yours realistic?” Perspectives on Psychological Science9(3): 305–318. 10.1177/1745691614528518
    https://doi.org/10.1177/1745691614528518 [Google Scholar]
  36. Tavakoli, Parveneh and Peter Skehan
    2005 Strategic planning, task structure and performance testing. InPlanning and task performance in a second language, ed. byRod Ellis, 239–273. Amsterdam: John Benjamins. 10.1075/lllt.11.15tav
    https://doi.org/10.1075/lllt.11.15tav [Google Scholar]
  37. Valentine, Jeffrey C.
    2011 “Replication in prevention science.” Prevention Science12, 103 (2011) 10.1007/s11121‑011‑0217‑6
    https://doi.org/10.1007/s11121-011-0217-6 [Google Scholar]
  38. Van Aert, Bobbie C. M. and Marcel A. L. M. van Assen
    2018 “Examining reproducibility in psychology: A hybrid method for combing a statistically significant original study and a replication.” Behavior Research Methods501: 1515–1539. 10.3758/s13428‑017‑0967‑6
    https://doi.org/10.3758/s13428-017-0967-6 [Google Scholar]
  39. Wang, Binhua and Li Tao
    2015 “An empirical study of pauses in Chinese-English simultaneous interpreting.” Perspectives23(1): 124–142. 10.1080/0907676X.2014.948885
    https://doi.org/10.1080/0907676X.2014.948885 [Google Scholar]
  40. Wen, Hao and Yanping Dong
    2019 “How does interpreting experience enhance working memory and short-term memory: A meta-analysis.” Journal of Cognitive Psychology31(8): 769–784. 10.1080/20445911.2019.1674857
    https://doi.org/10.1080/20445911.2019.1674857 [Google Scholar]
  41. Wright, Ben D. and John M. Linacre
    1994 “Reasonable mean-square fit values.” Rasch Measurement Transactions8(3): 370. https://rasch.org/rmt/rmt83b.htm
    [Google Scholar]
  42. Yang, Liu-yan
    2015 “An exploratory study of fluency in English output of Chinese consecutive interpreting learners.” Journal of Zhejiang International Studies University11, 60–68.
    [Google Scholar]
  43. Yu, Wenting and Vincent J. van Heuven
    2017 “Predicting judged fluency of consecutive interpreting from acoustic measures: Potential for automatic assessment and pedagogic implications.” Interpreting19(1): 47–68. 10.1075/intp.19.1.03yu
    https://doi.org/10.1075/intp.19.1.03yu [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error