image of Visual processing during computer-assisted consecutive interpreting
Buy:$35.00 + Taxes



This study investigates the visual processing patterns during computer-assisted consecutive interpreting (CACI). In phase I of the proposed CACI workflow, the interpreter listens to the source speech and respeaks it into speech recognition (SR) software. In phase II, the interpreter produces target speech supported by the SR text and its machine translation (MT) output. A group of students performed CACI with their eye movements tracked. In phase I, the participants devoted the majority of their attention to listening and respeaking, with very limited attention distributed to the SR text. However, a positive correlation was found between the percentage of dwell time on the SR text and the quality of respeaking, which suggests that active monitoring could be important. In phase II, the participants devoted more visual attention to the MT text than to the SR text and engaged in deeper and more effortful processing when reading the MT text. We identified a positive correlation between the percentage of dwell time on the MT text and interpreting quality in the L2–L1 direction but not in the L1–L2 direction. These results contribute to our understanding of computer-assisted interpreting and can provide insights for future research and training in this area.


Article metrics loading...

Loading full text...

Full text loading...


  1. Barr, D. J., Levy, R., Scheepers, C. & Tily, H. J.
    (2013) Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language (), –. 10.1016/j.jml.2012.11.001
    https://doi.org/10.1016/j.jml.2012.11.001 [Google Scholar]
  2. Bates, D., Mächler, M., Bolker, B. & Walker, S.
    (2015) Fitting linear mixed-effects models using lme4. Journal of Statistical Software, –. 10.18637/jss.v067.i01
    https://doi.org/10.18637/jss.v067.i01 [Google Scholar]
  3. Carl, M., Dragsted, B., Elming, J., Hardt, D. & Jakobsen, A. L.
    (2011) The process of post-editing: A pilot study. InA. L. Jakobsen, M. Carl, B. Sharp & M. Zock (Eds.), Proceedings of the 8th International NLPCS Workshop. Frederiksberg: Samfundslitteratur, –.
    [Google Scholar]
  4. Chafe, W.
    (1985) Linguistic differences produced by differences between speaking and writing. InD. R. Olson, A. Hildyard & N. Torrance (Eds.), Literacy, language, and learning: The nature and consequences of reading and writing. Cambridge: Cambridge University Press, –.
    [Google Scholar]
  5. Chen, S.
    (2017) Note-taking in consecutive interpreting: New data from pen recording. Translation and Interpreting (), –. 10.12807/ti.109201.2017.a02
    https://doi.org/10.12807/ti.109201.2017.a02 [Google Scholar]
  6. (2020a) The impact of directionality on the process and product in consecutive interpreting between Chinese and English: Evidence from pen recording and eye tracking. The Journal of Specialised Translation, –.
    [Google Scholar]
  7. (2020b) The process of note-taking in consecutive interpreting: A digital pen recording approach. Interpreting (), –. 10.1075/intp.00036.che
    https://doi.org/10.1075/intp.00036.che [Google Scholar]
  8. Chen, S. & Kruger, J.-L.
    (2023) The effectiveness of computer-assisted interpreting: A preliminary study based on English–Chinese consecutive interpreting. Translation and Interpreting Studies (), –. 10.1075/tis.21036.che
    https://doi.org/10.1075/tis.21036.che [Google Scholar]
  9. (under review). A computer-assisted consecutive interpreting mode: Training and evaluation.
    [Google Scholar]
  10. Chmiel, A., Janikowski, P. & Cieślewicz, A.
    (2020) The eye or the ear? Source language interference in sight translation and simultaneous interpreting. Interpreting (), –. 10.1075/intp.00043.chm
    https://doi.org/10.1075/intp.00043.chm [Google Scholar]
  11. Chmiel, A. & Lijewska, A.
    (2019) Syntactic processing in sight translation by professional and trainee interpreters: Professionals are more time-efficient while trainees view the source text less. Target (), –. 10.1075/target.18091.chm
    https://doi.org/10.1075/target.18091.chm [Google Scholar]
  12. (2022) Reading patterns, reformulation and eye-voice span (IVS) in sight translation. Translation and Interpreting Studies (), –. 10.1075/tis.21021.chm
    https://doi.org/10.1075/tis.21021.chm [Google Scholar]
  13. Cop, U., Drieghe, D. & Duyck, W.
    (2015) Eye movement patterns in natural reading: A comparison of monolingual and bilingual reading of a novel. PLoS ONE (), e0134008. 10.1371/journal.pone.0134008
    https://doi.org/10.1371/journal.pone.0134008 [Google Scholar]
  14. Dam, H. V. & Engberg, J.
    (2006) Assessing accuracy in consecutive interpreting: a comparison of semantic network analyses and intuitive assessments. InC. Heine, K. Schubert & H. Gerzymisch-Arbogast (Eds.), Text and translation: Theory and methodology of translation. Tübingen: Narr Francke Attempto Verlag, –.
    [Google Scholar]
  15. Davitti, E. & Sandrelli, A.
    (2020) Embracing the complexity: A pilot study on interlingual respeaking. Journal of Audiovisual Translation (), –. 10.47476/jat.v3i2.2020.135
    https://doi.org/10.47476/jat.v3i2.2020.135 [Google Scholar]
  16. Defrancq, B. & Fantinuoli, C.
    (2021) Automatic speech recognition in the booth: Assessment of system performance, interpreters’ performances and interactions in the context of numbers. Target (), –. 10.1075/target.19166.def
    https://doi.org/10.1075/target.19166.def [Google Scholar]
  17. Demareva, V. & Edeleva, Y.
    (2020) Eye-tracking based L2 detection: Universal and specific eye movement patterns in L1 and L2 reading. Procedia Computer Science, –. 10.1016/j.procs.2020.02.185
    https://doi.org/10.1016/j.procs.2020.02.185 [Google Scholar]
  18. Downie, J.
    (2023) Where is it all going? Technology, economic pressures and the future of interpreting. InG. Corpas Pastor & B. Defrancq (Eds.), Interpreting technologies – current and future trends. Amsterdam: John Benjamins, –. 10.1075/ivitra.37.11dow
    https://doi.org/10.1075/ivitra.37.11dow [Google Scholar]
  19. Dragsted, B. & Hansen, I. G.
    (2009) Exploring translation and interpreting hybrids. The case of sight translation. Meta (), –. 10.7202/038317ar
    https://doi.org/10.7202/038317ar [Google Scholar]
  20. Eskenazi, M. A.
    (2023) Best practices for cleaning eye movement data in reading research. Behavior Research Methods. 10.3758/s13428‑023‑02137‑x
    https://doi.org/10.3758/s13428-023-02137-x [Google Scholar]
  21. Fantinuoli, C.
    (2023) Towards AI-enhanced computer-assisted interpreting. InG. Corpas Pastor & B. Defrancq (Eds), Interpreting technologies – current and future trends. Amsterdam: John Benjamins, –. 10.1075/ivitra.37.03fan
    https://doi.org/10.1075/ivitra.37.03fan [Google Scholar]
  22. Gerver, D.
    (1969/2002) The effects of source language presentation rate on the performance of simultaneous conference interpreters. InF. Pöchhacker & M. Shlesinger (Eds.), The interpreting studies reader. London and New York: Routledge, –.
    [Google Scholar]
  23. Gile, D.
    (2009) Basic concepts and models for interpreter and translator training (revised edition). Amsterdam: John Benjamins. 10.1075/btl.8
    https://doi.org/10.1075/btl.8 [Google Scholar]
  24. Han, C.
    (2018) Latent trait modelling of rater accuracy of formative peer assessment of English-Chinese consecutive interpreting. Assessment & Evaluation in Higher Education(), –. 10.1080/02602938.2018.1424799
    https://doi.org/10.1080/02602938.2018.1424799 [Google Scholar]
  25. Ho, C-E.
    (2017) An integrated eye-tracking study into the cognitive process of English–Chinese sight translation: Impacts of training and experience. PhD dissertation, National Taiwan Normal University.
  26. Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H. & van de Weijer, J.
    (2011) Eye tracking: A comprehensive guide to methods and measures. Oxford: Oxford University Press.
    [Google Scholar]
  27. Huang, C-C.
    (2011) Tracking eye movements in sight translation. MA thesis, National Taiwan Normal University.
    [Google Scholar]
  28. Koglin, A.
    (2015) An empirical investigation of cognitive effort required to post-edit machine translated metaphors compared to the translation of metaphors. Translation and Interpreting (), –.
    [Google Scholar]
  29. Kuperman, V., Siegelman, N., Schroeder, S., Acartürk, C., Alexeeva, S., Amenta, S. … Usal, K. A.
    (2022) Text reading in English as a second language: Evidence from the Multilingual Eye-Movements Corpus. Studies in Second Language Acquisition (), –. 10.1017/S0272263121000954
    https://doi.org/10.1017/S0272263121000954 [Google Scholar]
  30. Lijewska, A., Chmiel, A. & Inhoff, A. W.
    (2022) Stages of sight translation: Evidence from eye movements. Applied Psycholinguistics, –. 10.1017/S014271642200025X
    https://doi.org/10.1017/S014271642200025X [Google Scholar]
  31. Liu, M. & Chiu, Y-H.
    (2009) Assessing source material difficulty for consecutive interpreting: Quantifiable measures and holistic judgment. Interpreting (), –. 10.1075/intp.11.2.07liu
    https://doi.org/10.1075/intp.11.2.07liu [Google Scholar]
  32. Moorkens, J., O’Brien, S., da Silva, I. A. L., Fonseca, N. B. d. L. & Alves, F.
    (2015) Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, –. 10.1007/s10590‑015‑9175‑2
    https://doi.org/10.1007/s10590-015-9175-2 [Google Scholar]
  33. O’Brien, S.
    (2011) Towards predicting post-editing productivity. Machine Translation (), –. 10.1007/s10590‑011‑9096‑7
    https://doi.org/10.1007/s10590-011-9096-7 [Google Scholar]
  34. Pan, J., Yan, M. & Laubrock, J.
    (2017) Perceptual span in oral reading: The case of Chinese. Scientific Studies of Reading (), –. 10.1080/10888438.2017.1283694
    https://doi.org/10.1080/10888438.2017.1283694 [Google Scholar]
  35. Parshina, O., Laurinavichyute, A. K. & Sekerina, I. A.
    (2021) Eye-movement benchmarks in Heritage Language reading. Bilingualism: Language and Cognition (), –. 10.1017/S136672892000019X
    https://doi.org/10.1017/S136672892000019X [Google Scholar]
  36. Pisani, E. & Fantinuoli, C.
    (2021) Measuring the impact of automatic speech recognition on number rendition in simultaneous interpreting. InC. Wang & B. Zheng (Eds.), Empirical studies of translation and interpreting. New York: Routledge, –. 10.4324/9781003017400‑14
    https://doi.org/10.4324/9781003017400-14 [Google Scholar]
  37. R Core Team
    R Core Team (2023) R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
    [Google Scholar]
  38. Rayner, K.
    (1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin (), –. 10.1037/0033‑2909.124.3.372
    https://doi.org/10.1037/0033-2909.124.3.372 [Google Scholar]
  39. (2009) Eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology (), –. 10.1080/17470210902816461
    https://doi.org/10.1080/17470210902816461 [Google Scholar]
  40. Rinne, J. O., Tommola, J., Laine, M., Krause, B. J., Schmidt, D., Kaasinen, V., Teräs, M., Sipilä, H. & Sunnari, M.
    (2000) The translating brain: Cerebral activation patterns during simultaneous interpreting. Neuroscience Letters (), –. 10.1016/S0304‑3940(00)01540‑8
    https://doi.org/10.1016/S0304-3940(00)01540-8 [Google Scholar]
  41. Romero-Fresco, P.
    (2011) Subtitling through speech recognition: Respeaking. Manchester: Routledge.
    [Google Scholar]
  42. (2015) Respeaking. InF. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies. Abingdon: Routledge, –.
    [Google Scholar]
  43. Romero-Fresco, P. & Martínez, J.
    (2015) Accuracy rate in live subtitling: The NER model. InR. Baños & J. Díaz-Cintas (Eds.), Audiovisual translation in a global context: Mapping an ever-changing landscape. London: Palgrave Macmillan, –. 10.1057/9781137552891_3
    https://doi.org/10.1057/9781137552891_3 [Google Scholar]
  44. Scott, S. D., Sharpe, H., O’Leary, K., Dehaeck, U., Hindmarsh, K., Moore, J. G. & Osmond, M. H.
    (2009) Court reporters: A viable solution for the challenges of focus group data collection?Qualitative Health Research (), –. 10.1177/1049732308327883
    https://doi.org/10.1177/1049732308327883 [Google Scholar]
  45. Shreve, G. M., Lacruz, I. & Angelone, E.
    (2010) Cognitive effort, syntactic disruption, and visual interference in a sight translation task. InG. M. Shreve & E. Angelone (Eds.). Translation and cognition. Amsterdam and Philadelphia: John Benjamins, –. 10.1075/ata.xv.05shr
    https://doi.org/10.1075/ata.xv.05shr [Google Scholar]
  46. Sun, F.
    (1993) Eye movements in reading Chinese: Paragraphs, single characters and pinyin. InS. F. Wright & R. Groner (Eds.), Facets of dyslexia and its remediation. Amsterdam: North-Holland, –. 10.1016/B978‑0‑444‑89949‑1.50023‑7
    https://doi.org/10.1016/B978-0-444-89949-1.50023-7 [Google Scholar]
  47. Sun, F., Morita, M. & Stark, L. W.
    (1985) Comparative patterns of reading eye movement in Chinese and English. Perception and Psychophysics, –. 10.3758/BF03204913
    https://doi.org/10.3758/BF03204913 [Google Scholar]
  48. Szarkowska, A., Dutka, Ł., Szychowska, A. & Pilipczuk, O.
    (2018) Visual attention distribution in intralingual respeaking: An eye-tracking study. InC. Walker & F. M. Federici (Eds.), Eye tracking and multidisciplinary studies on translation. Amsterdam: John Benjamins, –. 10.1075/btl.143.09sza
    https://doi.org/10.1075/btl.143.09sza [Google Scholar]
  49. Toral, A., Wieling, M. & Way, A.
    (2018) Post-editing effort of a novel with statistical and neural machine translation. Frontiers in Digital Humanities, –. 10.3389/fdigh.2018.00009
    https://doi.org/10.3389/fdigh.2018.00009 [Google Scholar]
  50. White, S. J., Warrington, K. L., McGowan, V. A. & Paterson, K. B.
    (2015) Eye movements during reading and topic scanning: Effects of word frequency. Journal of Experimental Psychology: Human Perception and Performance (), –.
    [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error