Volume 21, Issue 1
  • ISSN 1598-7647
  • E-ISSN: 2451-909X
Buy:$35.00 + Taxes



Due to the inaccuracies of Machine Translation (MT), Post-Editing (PE) is inevitable. This poses questions over whether human effort to polish an MT is worthwhile or whether it would be more efficient to translate manually. However, to date, fewer attempts have been made to compare the cognitive effort in the PE process and the sub-phases (orientation, drafting, and revision) of PE with that of Human Translation (HT). To fill this gap, the current study aims to investigate and compare cognitive effort in HT and PE processes in translation from Chinese to English. Data were collected via eye-tracking and keyboard-logging approaches from 25 participants recruited to fulfil three HT and three PE tasks respectively. The comparison of cognitive effort was made from the processes of HT and PE, and their different sub-phases. The study reveals a significant difference in cognitive effort, orientation duration, and drafting duration between HT and PE.


Article metrics loading...

Loading full text...

Full text loading...


  1. Alves, Fabio, Arlene Koglin, Bartolomé Mesa-Lao, Mercedes García Martínez, Norma B. de Lima Fonseca, Arthur de Melo Sá, José Luiz Gonçalves, Karina Sarto Szpak, Kyoko Sekino, and Marceli Aquino
    2016 “Analysing the Impact of Interactive Machine Translation on Post-Editing Effort.” InNew Directions in Empirical Translation Process Research, edited byMichael Carl, Srinivas Bangalore, and Moritz Schaeffer, 1st ed., 77–94. Cham: Springer. 10.1007/978‑3‑319‑20358‑4_4
    https://doi.org/10.1007/978-3-319-20358-4_4 [Google Scholar]
  2. Bradley, James V.
    1958 “Complete Counterbalancing of Immediate Sequential Effects in a Latin Square Design.” Journal of the American Statistical Association53 (282): 525–28. 10.1080/01621459.1958.10501456
    https://doi.org/10.1080/01621459.1958.10501456 [Google Scholar]
  3. Carl, Michael
    2012 “The CRITT TPR-DB 1.0: A Database for Empirical Human Translation Process Research.” InWorkshop on Post-Editing Technology and Practice, 9–18. San Diego, California, USA: Association for Machine Translation in the Americas. https://aclanthology.org/2012.amta-wptp.1
    [Google Scholar]
  4. Carl, Michael, Barbara Dragsted, Jakob Elming, Daniel Hardt, and Arnt Lykke Jakobsen
    2011 “The Process of Post-Editing: A Pilot Study.” Copenhagen Studies in Language411: 131–42.
    [Google Scholar]
  5. Carl, Michael, Barbara Dragsted, and Arnt Lykke Jakobsen
    2011 “On the Systematicity of Human Translation Processes.” InTralogy 2011. Translation Careers and Technologies: Convergence Points for the Future. Paris, France. lodel.irevues.inist.fr/tralogy/index.php?id=103
    [Google Scholar]
  6. Carl, Michael, and Martin Kay
    2011 “Gazing and Typing Activities during Translation: A Comparative Study of Translation Units of Professional and Student Translators.” Meta56 (4): 952–75. 10.7202/1011262ar
    https://doi.org/10.7202/1011262ar [Google Scholar]
  7. Conklin, Kathy, Ana Pellicer-Sánchez, and Gareth Carrol
    2018Eye-Tracking: A Guide for Applied Linguistics Research. Cambridge, UK: Cambridge University Press. 10.1017/9781108233279
    https://doi.org/10.1017/9781108233279 [Google Scholar]
  8. Cumbreno, Cristina, and Nora Aranberri
    2019 “Comparison of Temporal, Technical and Cognitive Dimension Measurements for Post-Editing Effort.” InSecond MEMENTO Workshop on Modelling Parameters of Cognitive Effort in Translation Production, 5–6. Dublin, Ireland.
    [Google Scholar]
  9. Daems, Joke, Sonia Vandepitte, Robert J. Hartsuiker, and Lieve Macken
    2017 “Translation Methods and Experience: A Comparative Analysis of Human Translation and Post-Editing with Students and Professional Translators.” Meta62 (2): 245–70. 10.7202/1041023ar
    https://doi.org/10.7202/1041023ar [Google Scholar]
  10. Dragsted, Barbara
    2010 “Coordination of Reading and Writing Processes in Translation.” InTranslation and Cognition, edited byGregory M. Shreve and Erik Angelone, 41–61. Philadelphia, USA: John Benjamins Publishing. 10.1075/ata.xv.04dra
    https://doi.org/10.1075/ata.xv.04dra [Google Scholar]
  11. Dragsted, Barbara, and Michael Carl
    2013 “Towards a Classification of Translation Styles Based on Eye-Tracking and Keylogging Data.” Journal of Writing Research5 (1): 133–58. 10.17239/jowr‑2013.05.01.6
    https://doi.org/10.17239/jowr-2013.05.01.6 [Google Scholar]
  12. Durban, Chris
    2011 “Translation – Getting It Right. A Guide to Buying Translation.” American Translators Association. AccessedApril 22, 2022. www.atanet.org
    [Google Scholar]
  13. Ferreira, Aline, John Wayne Schwieter, Alexandra Gottardo, and Jefferey Jones
    2016 “Cognitive Effort in Direct and Inverse Translation Performance: Insight from Eye-Tracking Technology.” Cadernos de Tradução36 (3): 60–80. 10.5007/2175‑7968.2016v36n3p60
    https://doi.org/10.5007/2175-7968.2016v36n3p60 [Google Scholar]
  14. Green, Spence, Jeffrey Heer, and Christopher D. Manning
    2013 “The Efficacy of Human Post-Editing for Language Translation.” InSIGCHI Conference on Human Factors in Computing Systems, 439–48. Paris, France. 10.1145/2470654.2470718
    https://doi.org/10.1145/2470654.2470718 [Google Scholar]
  15. Guerberof-Arenas, Ana
    2012 Productivity and Quality in the Post-Editing of Outputs from Translation Memories and Machine Translation. PhD diss. Universitat Rovira I Virgili.
  16. 2014 “The Role of Professional Experience in Post-Editing from a Quality and Productivity Perspective.” InPost-Editing of Machine Translation: Processes and Applications, edited byLaura Winther Balling, Michael Carl, Sharon O’Brien, Michel Simard, and Specia Lucia, 51–77. Newcastle-upon-Tyne, UK: Cambridge Scholars Publishing.
    [Google Scholar]
  17. Hvelplund, Kristian Tangsgaard
    2011Allocation of Cognitive Resources in Translation. An Eye-Tracking and Key-Logging Study.
    [Google Scholar]
  18. 2014 “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-Tracking Data.” MonTI Special Issue – Minding Translation, 201–23. 10.6035/MonTI.2014.ne1.6
    https://doi.org/10.6035/MonTI.2014.ne1.6 [Google Scholar]
  19. Jakobsen, Arnt Lykke
    2002 “Translation Drafting by Professional Translators and by Translation Students.” Copenhagen Studies in Language271: 191–204.
    [Google Scholar]
  20. Jia, Yanfang, Michael Carl, and Xiangling Wang
    2019 “Post-Editing Neural Machine Translation versus Phrase-Based Machine Translation for English–Chinese.” Machine Translation33 (1): 9–29. 10.1007/s10590‑019‑09229‑6
    https://doi.org/10.1007/s10590-019-09229-6 [Google Scholar]
  21. Just, M., and P. Carpenter
    1976 “Eye Fixations and Cognitive Processes.” Cognitive Psychology, 8 (4): 441–80. 10.1016/0010‑0285(76)90015‑3
    https://doi.org/10.1016/0010-0285(76)90015-3 [Google Scholar]
  22. Just, Marcel Adam, and Patricia A. Carpenter
    1980 “A Theory of Reading: From Eye Fixations to Comprehension.” Psychological Review871: 329–54. 10.1037/0033‑295X.87.4.329
    https://doi.org/10.1037/0033-295X.87.4.329 [Google Scholar]
  23. Koglin, Arlene
    2015 “An Empirical Investigation of Cognitive Effort Required to Post-Edit Machine Translated Metaphors Compared to the Translation of Metaphors.” Translation & Interpreting7 (1): 126–41. 10.12807/ti.106201.2015.a06
    https://doi.org/10.12807/ti.106201.2015.a06 [Google Scholar]
  24. Koglin, Arlene, and Rossana Cunha
    2019 “Investigating the Post-Editing Effort Associated with Machine-Translated Metaphors: A Process-Driven Analysis.” Journal of Specialised Translation311: 38–59.
    [Google Scholar]
  25. Koponen, Maarit
    2012 “Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations.” InSeventh Workshop on Statistical Machine Translation, 181–90. Montreal, Canada: Association for Computational Linguistics. www.aclweb.org/anthology/W12-3123
    [Google Scholar]
  26. 2016 “Is Machine Translation Post-Editing Worth the Effort? A Survey of Research into Post-Editing and Effort.” Journal of Specialised Translation251: 131–48.
    [Google Scholar]
  27. Koponen, Maarit, Wilker Aziz, Luciana Ramos, and Lucia Specia
    2012 “Post-editing time as a measure of cognitive effort.“ InAMTA 2012 Workshop on Post-editing Technology and Practice (WPTP). San Diego, California: Association for Machine Translation in the Americas. https://aclanthology.org/2012.amta-wptp.2
    [Google Scholar]
  28. Krings, Hans P.
    2001Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes. Kent, Ohio, USA: Kent State University Press.
    [Google Scholar]
  29. Lacruz, Isabel, Michael Carl, Masaru Yamada, and Akiko Aizawa
    2016 “Pause Metrics and Machine Translation Utility.” InThe 22nd Annual Meeting of the Association for Natural Language Processing, NLP 2016, 1213–16. Sendai, Japan: The Association for Natural Language Processing.
    [Google Scholar]
  30. Lacruz, Isabel, Michael Denkowski, and Alon Lavie
    2014 “Cognitive Demand and Cognitive Effort in Post-Editing.” InProceedings of the 11th Conference of the Association for Machine Translation in the Americas, 73–84. Vancouver, Canada: Association for Machine Translation in the Americas. https://aclanthology.org/2014.amta-wptp.6
    [Google Scholar]
  31. Lacruz, Isabel, and Gregory M. Shreve
    2014 “Pauses and Cognitive Effort in Post-Editing.” InPost-Editing of Machine Translation: Processes and Applications, edited byLaura Winther Balling, Michael Carl, and Sharon O’Brien, 246–73. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.
    [Google Scholar]
  32. Lacruz, Isabel, Gregory M. Shreve, and Erik Angelone
    2012 “Average Pause Ratio as an Indicator of Cognitive Effort in Post-Editing: A Case Study.” InWorkshop on Post-Editing Technology and Practice. San Diego, California, USA: Association for Machine Translation in the Americas. https://aclanthology.org/2012.amta-wptp.3
    [Google Scholar]
  33. Läubli, Samuel, Chantal Amrhein, Patrick Düggelin, Beatriz Gonzalez, Alena Zwahlen, and Martin Volk
    2019 “Post-Editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain.” ArXiv Preprint. arxiv.org/abs/1906.01685.
    [Google Scholar]
  34. Martín, Ricardo Muñoz
    2010 “On Paradigms and Cognitive Translatology.” InTranslation and Cognition, edited byGregory M. Shreve and Erik Angelone, 169–87. Philadelphia, Pa, USA: John Benjamins Publishing. 10.1075/ata.xv.10mun
    https://doi.org/10.1075/ata.xv.10mun [Google Scholar]
  35. Mesa-Lao, Bartolomé
    2014 “Gaze Behaviour on Source Texts: An Exploratory Study Comparing Translation and Post-Editing.” InPost-Editing of Machine Translation: Processes and Applications, 219–45. Cambridge, UK: Cambridge Scholars Publishing.
    [Google Scholar]
  36. Nitzke, Jean, and Katharina Oster
    2016 “Comparing Translation and Post-Editing: An Annotation Schema for Activity Units.” InNew Directions in Empirical Translation Process Research, edited byMichael Carl, Srinivas Bangalore, and Moritz Schaeffer, 293–308. 10.1007/978‑3‑319‑20358‑4_14
    https://doi.org/10.1007/978-3-319-20358-4_14 [Google Scholar]
  37. O’Brien, Sharon
    2006 “Pauses as Indicators of Cognitive Effort in Post-Editing Machine Translation Output.” Across Languages and Cultures7 (1): 1–21. 10.1556/Acr.7.2006.1.1
    https://doi.org/10.1556/Acr.7.2006.1.1 [Google Scholar]
  38. 2007 “An Empirical Investigation of Temporal and Technical Post-Editing Effort.” Translation and Interpreting Studies2 (1): 83–136. 10.1075/tis.2.1.03ob
    https://doi.org/10.1075/tis.2.1.03ob [Google Scholar]
  39. Pellicer-Sánchez, Ana
    2016 “Incidental L2 Vocabulary Acquisition from and While Reading: An Eye-Tracking Study.” Studies in Second Language Acquisition381: 97–130. 10.1017/S0272263115000224
    https://doi.org/10.1017/S0272263115000224 [Google Scholar]
  40. Plitt, Mirko, and François Masselot
    2010 “A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context.” The Prague Bulletin of Mathematical Linguistics931: 7–16. 10.2478/v10108‑010‑0010‑x
    https://doi.org/10.2478/v10108-010-0010-x [Google Scholar]
  41. R Core Team
    R Core Team 2021 “R: A Language and Environment for Statistical Computing.” Vienna, Austria: R Foundation for Statistical Computing. https://www.r-project.org/
  42. Richardson, John T. E.
    2018 “The Use of Latin-Square Designs in Educational and Psychological Research.” Educational Research Review241: 84–97. 10.1016/j.edurev.2018.03.003
    https://doi.org/10.1016/j.edurev.2018.03.003 [Google Scholar]
  43. Sanchez-Torron, Marina, and Philipp Koehn
    2016 “Machine Translation Quality and Post-Editor Productivity.” InAMTA 2016 – The Twelfth Conference of the Association for Machine Translation in the Americas, edited bySpence Green and Lane Schwartz, 11:16–26. Austin, Texas, USA: Association for Machine Translation in the Americas. www.amtaweb.org/amta-2016-in-austin-tx
    [Google Scholar]
  44. Schaeffer, Moritz, Michael Carl, Isabel Lacruz, and Akiko Aizawa
    2016 “Measuring Cognitive Translation Effort with Activity Units.” Baltic Journal of Modern Computing4 (2): 331–45.
    [Google Scholar]
  45. Screen, Benjamin
    2017 “Productivity and Quality When Editing Machine Translation and Translation Memory Outputs: An Empirical Analysis of English to Welsh Translation.” Studia Celtica Posnaniensia2 (1): 119–42. 10.1515/scp‑2017‑0007
    https://doi.org/10.1515/scp-2017-0007 [Google Scholar]
  46. Sjørup, Annette Camilla
    2013 Cognitive Effort in Metaphor Translation: An Eye-Tracking and Key-Logging Study. PhD diss. Copenhagen Business School.
  47. Sousa, Sheila C. M. de, Wilker Aziz, and Lucia Specia
    2011 “Assessing the Post-Editing Effort for Automatic and Semi-Automatic Translations of DVD Subtitles.” InRecent Advances in Natural Language Processing, edited byRuslan Mitkov and Galia Angelova, 97–103. Hissar, Bulgaria.
    [Google Scholar]
  48. Stasimioti, Maria, and Vilelmini Sosoni
    2020 “Translation vs Post-Editing of NMT Output: Measuring Effort in the English-Greek Language Pair.” In14th Conference of the Association for Machine Translation in the Americas,1st Workshop on Post-Editing in Modern-Day Translation, 109–24.
    [Google Scholar]
  49. 2021 “Investigating Post-Editing: A Mixed-Methods Study with Experienced and Novice Translators in the English-Greek Language Pair.” InTranslation, Interpreting, Cognition: The Way out of the Box, edited byTra&Co Group, 79–104. Berlin, Germany: Language Science Press. 10.5281/zenodo.4545037
    https://doi.org/10.5281/zenodo.4545037 [Google Scholar]
  50. Sun, Sanjun, and Gregory M. Shreve
    2014 “Measuring Translation Difficulty.” Target26 (1): 98–127. 10.1075/target.26.1.04sun
    https://doi.org/10.1075/target.26.1.04sun [Google Scholar]
  51. Sung, Yao-Ting, Tao-Hsing Chang, Wei-Chun Lin, Kuan-Sheng Hsieh, and Kuo-En Chang
    2016 “CRIE: An Automated Analyzer for Chinese Texts.” Behavior Research Methods48 (4): 1238–51. 10.3758/s13428‑015‑0649‑1
    https://doi.org/10.3758/s13428-015-0649-1 [Google Scholar]
  52. Vanroy, Bram, Orphée De Clercq, and Lieve Macken
    2019 “Correlating Process and Product Data to Get an Insight into Translation Difficulty.” Perspectives: Studies in Translation Theory and Practice27 (6): 924–41. 10.1080/0907676X.2019.1594319
    https://doi.org/10.1080/0907676X.2019.1594319 [Google Scholar]
  53. Vieira, Lucas Nunes
    2014 “Indices of Cognitive Effort in Machine Translation Post-Editing.” Machine Translation281: 187–216. 10.1007/s10590‑014‑9156‑x
    https://doi.org/10.1007/s10590-014-9156-x [Google Scholar]
  54. Whyatt, Bogusława
    2019 “In Search of Directionality Effects in the Translation Process and in the End Product.” Translation, Cognition & Behavior2 (1): 79–100. 10.1075/tcb.00020.why
    https://doi.org/10.1075/tcb.00020.why [Google Scholar]
  55. Whyatt, Bogusława, Katarzyna Stachowiak, and Marta Kajzer-Wietrzny
    2016 “Similar and Different: Cognitive Rhythm and Effort in Translation and Paraphrasing.” Poznan Studies in Contemporary Linguistics52 (2): 175–208. 10.1515/psicl‑2016‑0007
    https://doi.org/10.1515/psicl-2016-0007 [Google Scholar]
  56. Whyatt, Bogusława, Olga Witczak, and Ewa Tomczak
    2021 “Information Behaviour in Bidirectional Translators: Focus on Online Resources.” Interpreter and Translator Trainer15 (2): 154–71. 10.1080/1750399X.2020.1856023
    https://doi.org/10.1080/1750399X.2020.1856023 [Google Scholar]
  57. Zhechev, Ventsislav
    2012 “Machine Translation Infrastructure and Post-Editing Performance at Autodesk.” InWorkshop on Post-Editing Technology and Practice. San Diego, California: Association for Machine Translation in the Americas. https://aclanthology.org/2012.amta-wptp.10
    [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error