Volume 6, Issue 1
  • ISSN 2542-5277
  • E-ISSN: 2542-5285



This paper presents a user study with 15 professional translators in the English-Spanish combination. We present the concept of Machine Translation User Experience (MTUX) and compare the effects of traditional post-editing (TPE) and interactive post-editing (IPE) on MTUX, translation quality and productivity. Results suggest that translators prefer IPE to TPE because they are in control of the interaction in this new form of translator-computer interaction and feel more empowered in their interaction with Machine Translation. Productivity results also suggest that IPE may be an interesting alternative to TPE, given the fact that translators worked faster in IPE even though they had no experience in this new machine translation post-editing modality, but were already used to TPE.

Available under the CC BY 4.0 license.

Article metrics loading...

Loading full text...

Full text loading...



  1. Alabau Gonzalvo, Vicent, Ragnar Bonkb, Christian Buck, Michael Carlb, Francisco Casacuberta Nolla, Mercedes García Martínez, Jesús González Rubio, Philip Koehn, Luis Alberto Leiva Torres, Bartolomé Mesa Lao, Daniel Ortiz Martínez, Herve Saint-Amand, Germán Sanchis Trilles & Chara Tsoukalak
    2013 CASMACAT: An open source workbench for advanced computer aided translation. Prague Bulletin of Mathematical Linguistics, 1001: 101–12. De Gruyter Open. 10.2478/pralin‑2013‑0016
    https://doi.org/10.2478/pralin-2013-0016 [Google Scholar]
  2. Alabau Gonzalvo, Vicent, Michael Carl, Mercedes García Martínez & Jesús González Rubio
    2016 Learning Advanced Post-Editing. InNew Directions in Empirical Translation Process Research. Edited byMichael Carl, Srinivas Bangalore, Moritz Schaeffer, 95–110. Cham: Springer. 10.1007/978‑3‑319‑20358‑4_5
    https://doi.org/10.1007/978-3-319-20358-4_5 [Google Scholar]
  3. Alves, Fabio, Karina Sarto Szpak, José Luiz Gonçalves, Kyoko Sekino, Marceli Aquino, Rodrigo Araújo e Castro, Arlene Koglin, Norma B. de Lima Fonseca, and Bartolomé Mesa Lao
    2016 Investigating Cognitive Effort in Post-Editing: A Relevance-Theoretical Approach. InEyetracking and Applied Linguistics. Edited bySilvia Hansen-Schirra & Sambor Grucza, 109–42. Language Science Press. langsci-press.org/catalog/book/108
    [Google Scholar]
  4. Artstein, Ron
    2017 Inter-Annotator Agreement. InHandbook of Linguistic Annotation. Edited byNancy Ide & James Pustejovsky, 297–313. Dordrecht: Springer Netherlands. 10.1007/978‑94‑024‑0881‑2_11
    https://doi.org/10.1007/978-94-024-0881-2_11 [Google Scholar]
  5. Bertoldi, Nicola, Davide Caroselli & Marcello Federico
    2018 The ModernMT Project. InProceedings of the 21st Annual Conference of the European Association for Machine Translation. Edited byJuan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Maja Popovic, Celia Rico, André Martins, Joachim Van den Bogaert, Mikel L. Forcada, 345–347. Alicante: European Association for Machine Translation. rua.ua.es/dspace/handle/10045/76096
    [Google Scholar]
  6. Bowker, Lynne & Jairo Buitrago Ciro
    2018 Localizing Websites Using Machine Translation: Exploring Connections between User EXperience and Translatability. InThe Human Factor in Machine Translation. Edited bySin-wai Chan, 29–52. London: Routledge. 10.4324/9781315147536‑2
    https://doi.org/10.4324/9781315147536-2 [Google Scholar]
  7. Briva-Iglesias, Vicent
    2021 Traducción humana vs. traducción automática: análisis contrastivo e implicaciones para la aplicación de la traducción automática en traducción jurídica. Mutatis Mutandis. Revista Latinoamericana de Traducción14 (2): 571–600. 10.17533/udea.mut.v14n2a14
    https://doi.org/10.17533/udea.mut.v14n2a14 [Google Scholar]
  8. 2022 English-Catalan Neural Machine Translation: State-of-the-Art Technology, Quality, and Productivity. Tradumàtica: Tecnologies de La Traducció201: 149–76. 10.5565/rev/tradumatica.303
    https://doi.org/10.5565/rev/tradumatica.303 [Google Scholar]
  9. Briva-Iglesias, Vicent, Sharon O’Brien & Benjamin R. Cowan
    2023 Measuring Machine Translation User Experience: A Comparison between AttrakDiff and User Experience Questionnaire. InProceedings of the 24th Annual Conference of the European Association for Machine Translation.
    [Google Scholar]
  10. . Forthcoming. Translators’ Pre-Task Perceptions of CAT Tools and MTPE, and Their Relationship with Translation Quality: Implications for Training.
    [Google Scholar]
  11. Cadwell, Patrick, Sheila Castilho, Sharon O’Brien & Linda Mitchell
    2016 Human Factors in Machine Translation and Post-Editing among Institutional Translators. Translation Spaces5 (2): 222–43. 10.1075/ts.5.2.04cad
    https://doi.org/10.1075/ts.5.2.04cad [Google Scholar]
  12. Cadwell, Patrick, Sharon O’Brien & Carlos S. C. Teixeira
    2018 Resistance and Accommodation: Factors for the (Non-) Adoption of Machine Translation among Professional Translators. Perspectives26 (3): 301–21. 10.1080/0907676X.2017.1337210
    https://doi.org/10.1080/0907676X.2017.1337210 [Google Scholar]
  13. Castilho, Sheila
    2016 Measuring Acceptability of Machine Translated Enterprise Content. PhD Thesis, Dublin City University. Doctoral, Dublin City University. Faculty of Humanities and Social Sciences. doras.dcu.ie/21342/
    [Google Scholar]
  14. Castilho, Sheila, Stephen Doherty, Federico Gaspari & Joss Moorkens
    2018 Approaches to Human and Machine Translation Quality Assessment. InTranslation Quality Assessment: From Principles to Practice. Edited byJoss Moorkens, Sheila Castilho, Federico Gaspari & Stephen Doherty, 9–38. Machine Translation: Technologies and Applications. Cham: Springer International Publishing. 10.1007/978‑3‑319‑91241‑7_2
    https://doi.org/10.1007/978-3-319-91241-7_2 [Google Scholar]
  15. Daems, Joke & Lieve Macken
    2019 Interactive Adaptive SMT versus Interactive Adaptive NMT: A User Experience Evaluation. Machine Translation33 (1): 117–34. 10.1007/s10590‑019‑09230‑z
    https://doi.org/10.1007/s10590-019-09230-z [Google Scholar]
  16. Dix, Alan
    2010 Human-Computer Interaction: A Stable Discipline, a Nascent Science, and the Growth of the Long Tail. Interacting with Computers22 (1): 13–27. 10.1016/j.intcom.2009.11.007
    https://doi.org/10.1016/j.intcom.2009.11.007 [Google Scholar]
  17. Esteban Lauzán, José, José Lorenzo Mon, Antonio Sánchez Valderrábanos & Guy Lapalme
    2004 TransType2: An Innovative Computer-Assisted Translation System. InProceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions, 94–97. Barcelona, Spain: Association for Computational Linguistics. 10.3115/1219044.1219045
    https://doi.org/10.3115/1219044.1219045 [Google Scholar]
  18. Etchegoyhen, Thierry, Anna Fernández Torné, Andoni Azpeitia Zaldua, Eva Martínez García & Anna Matamala Ripoll
    2018 Evaluating Domain Adaptation for Machine Translation Across Scenarios. InProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Miyazaki, Japan: European Language Resources Association. https://www.aclweb.org/anthology/L18-1002
    [Google Scholar]
  19. Forlizzi, Jodi & Katja Battarbee
    2004 Understanding Experience in Interactive Systems. InProceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, 261–68. DIS ’04. New York, NY, USA: Association for Computing Machinery. 10.1145/1013115.1013152
    https://doi.org/10.1145/1013115.1013152 [Google Scholar]
  20. Gaspari, Federico, Antonio Toral Ruiz, Sudip Kumar Naskar, Declan Groves & Andy Way
    2014 Perception vs Reality: Measuring Machine Translation Post-Editing Productivity. InProceedings of the 11th Conference of the Association for Machine Translation in the Americas, 60–72. Vancouver, Canada: Association for Machine Translation in the Americas.
    [Google Scholar]
  21. Green, Spence
    2016 Interactive Machine Translation. InConferences of the Association for Machine Translation in the Americas. Invited talk.
    [Google Scholar]
  22. Green, Spence, Jason Chuang, Jeffrey Heer & Christopher D. Manning
    2014 Predictive Translation Memory: A Mixed-Initiative System for Human Language Translation. InProceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, 177–187. Honolulu Hawaii USA: ACM. 10.1145/2642918.2647408
    https://doi.org/10.1145/2642918.2647408 [Google Scholar]
  23. Green, Spence, Sida I. Wang, Jason Chuang, Jeffrey Heer, Sebastian Schuster & Christopher D. Manning
    2014 Human Effort and Machine Learnability in Computer Aided Translation. InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1225–1236. Doha, Qatar: Association for Computational Linguistics. 10.3115/v1/D14‑1130
    https://doi.org/10.3115/v1/D14-1130 [Google Scholar]
  24. Guerberof Arenas, Ana, Joss Moorkens & Sharon O’Brien
    2021 The Impact of Translation Modality on User Experience: An Eye-Tracking Study of the Microsoft Word User Interface. Machine Translation35 (2): 205–37. 10.1007/s10590‑021‑09267‑z
    https://doi.org/10.1007/s10590-021-09267-z [Google Scholar]
  25. Guerberof Arenas, Ana
    2008 Productivity and Quality in the Post-Editing of Outputs from Translation Memories and Machine Translation. Localisation Focus The International Journal of Localisation, 7 (1), 11–21.
    [Google Scholar]
  26. ISO
    ISO 2018 ISO 9241-11:2018(En), Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts 2018 https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
  27. Karakanta, Alina, Luisa Bentivogli, Mauro Cettolo, Matteo Negri & Marco Turchi
    2022 Post-Editing in Automatic Subtitling: A Subtitlers’ Perspective. InProceedings of the 23rd Annual Conference of the European Association for Machine Translation, 261–70. Ghent, Belgium: European Association for Machine Translation. https://aclanthology.org/2022.eamt-1.29
    [Google Scholar]
  28. Knowles, Rebecca, Marina Sánchez Torrón & Philipp Koehn
    2019 Neural Interactive Translation Prediction. Machine Translation331: 135–134. 10.1007/s10590‑019‑09235‑8
    https://doi.org/10.1007/s10590-019-09235-8 [Google Scholar]
  29. Koehn, Philipp
    2009 A Web-Based Interactive Computer Aided Translation Tool. InProceedings of the ACL-IJCNLP 2009 Software Demonstrations. Edited byGary Geunbae Lee, Sabine Schulte im Walde, 17–20. Suntec, Singapore: Association for Computational Linguistics. https://www.aclweb.org/anthology/P09-4005. 10.3115/1667872.1667877
    https://doi.org/10.3115/1667872.1667877 [Google Scholar]
  30. Koehn, Philipp, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, Evan Herbst, Hieu Hoang
    , et al 2007 Moses: Open Source Toolkit for Statistical Machine Translation. InProceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions—ACL. Edited bySophia Ananiadou, 177–180. Prague: Association for Computational Linguistics. 10.3115/1557769.1557821
    https://doi.org/10.3115/1557769.1557821 [Google Scholar]
  31. Koponen, Maarit
    2012 Comparing Human Perceptions of Post-Editing Effort with Post-Editing Operations. InProceedings of the Seventh Workshop on Statistical Machine Translation. Edited byChris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut & Lucia Specia, 181–190. Montréal, Canada: Association for Computational Linguistics.
    [Google Scholar]
  32. Koponen, Maarit, Umut Sulubacak, Kaisa Vitikainen & Jörg Tiedemann
    2020 MT for Subtitling: Investigating Professional Translators’ User Experience and Feedback. InProceedings of 1st Workshop on Post-Editing in Modern-Day Translation. Edited byJohn E. Ortega, Marcello Federico, Constantin Orasan, Maja Popovic, 79–92. Virtual: AMTA. https://aclanthology.org/2020.amta-pemdt.6
    [Google Scholar]
  33. Langlais, Philippe, George Foster & Guy Lapalme
    2000 TransType: A Computer-Aided Translation Typing System. InANLP-NAACL 2000 Workshop: Embedded Machine Translation Systems, 1–6. https://www.aclweb.org/anthology/W00-0507
    [Google Scholar]
  34. Läubli, Samuel, Sheila Castilho, Graham Neubig, Rico Sennrich, Qinlan Shen & Antonio Toral Ruiz
    2020 A Set of Recommendations for Assessing Human–Machine Parity in Language Translation. Journal of Artificial Intelligence Research671 (March): 653–672–653–72. 10.1613/jair.1.11371
    https://doi.org/10.1613/jair.1.11371 [Google Scholar]
  35. Läubli, Samuel & Spence Green
    2019 Translation Technology Research and Human–Computer Interaction. InThe Routledge Handbook of Translation and Technology, edited byMinako O’Hagan, 370–83. New York, NY, USA: Routledge. 10.4324/9781315311258‑22
    https://doi.org/10.4324/9781315311258-22 [Google Scholar]
  36. Läubli, Samuel, Patrick Simianer, Joern Wuebker, Geza Kovacs, Rico Sennrich & Spence Green
    2022 The Impact of Text Presentation on Translator Performance. Target. International Journal of Translation Studies34 (2): 309–342. arxiv.org/abs/2011.05978. 10.1075/target.20006.lau
    https://doi.org/10.1075/target.20006.lau [Google Scholar]
  37. Laugwitz, Bettina, Theo Held & Martin Schrepp
    2008 Construction and Evaluation of a User Experience Questionnaire. International Journal of Interactive Multimedia and Artificial Intelligence4 (4): 63–76. 10.1007/978‑3‑540‑89350‑9_6
    https://doi.org/10.1007/978-3-540-89350-9_6 [Google Scholar]
  38. Macklovitch, Elliott
    2006 TransType2: The Last Word. InProceedings of the 5th Edition of the International Conference on Language Resources and Evaluation. Edited byNicoletta Calzolari, Khalid Choukri, Aldo Gangemi, Bente Maegaard, Joseph Mariani, Jan Odijk, Daniel Tapias, 1–6. Genoa, Italy: International Conference on Language Resources and Evaluation (LREC).
    [Google Scholar]
  39. Martikainen, Hanna
    2022 Ghosts in the Machine: Can Adaptive MT Help Reclaim a Place for the Human in the Loop?. InPortail HAL Sorbonne Nouvelle. https://hal.archives-ouvertes.fr/hal-03548696
    [Google Scholar]
  40. Moorkens, Joss
    2020 “A Tiny Cog in a Large Machine”: Digital Taylorism in the Translation Industry. Translation Spaces9 (1): 12–34. 10.1075/ts.00019.moo
    https://doi.org/10.1075/ts.00019.moo [Google Scholar]
  41. Nurminen, Mary
    2019 Decision-Making, Risk, and Gist Machine Translation in the Work of Patent Professionals. InProceedings of the 8th Workshop on Patent and Scientific Literature Translation, 32–42. Dublin: European Association for Machine Translation. https://aclanthology.org/W19-7204
    [Google Scholar]
  42. Nurminen, Mary & Niko Papula
    2018 Gist MT Users: A Snapshot of the Use and Users of One Online MT Tool. InEdited byJuan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Miquel Esplà-Gomis, Maja Popovic, Celia Rico, André Martins, Joachim Van den Bogaert, Mikel L. Forcada, 199–208. Alicante: European Association for Machine Translation. rua.ua.es/dspace/handle/10045/76049
  43. O’Brien, Sharon
    2006 Pauses as Indicators of Cognitive Effort in Post-Editing Machine Translation Output. Across Languages and Cultures7 (1): 1–21. 10.1556/Acr.7.2006.1.1
    https://doi.org/10.1556/Acr.7.2006.1.1 [Google Scholar]
  44. 2022 How to Deal with Errors in Machine Translation: Post-Editing. InMachine Translation for Everyone, 105–20. Berlin: Language Science Press. CitetononCRdoi:10.5281/zenodo.6759982
    https://doi.org/Cite to nonCR doi: 10.5281/zenodo.6759982 [Google Scholar]
  45. O’Brien, Sharon, Maureen Ehrensberger-Dow, Marcel Hasler & Megan Connolly
    2017 Irritating CAT Tool Features That Matter to Translators. Hermes: Journal of Language and Communication in Business561: 145–62.
    [Google Scholar]
  46. Olohan, Maeve
    2011 Translators and Translation Technology: The Dance of Agency. Translation Studies4 (3): 342–57. 10.1080/14781700.2011.589656
    https://doi.org/10.1080/14781700.2011.589656 [Google Scholar]
  47. Paz, Freddy & Jose Pow-Sang
    2016 A Systematic Mapping Review of Usability Evaluation Methods for Software Development Process101: 165–78. 10.14257/ijseia.2016.10.1.16
    https://doi.org/10.14257/ijseia.2016.10.1.16 [Google Scholar]
  48. Pérez Macías, Lorena, María del Mar Sánchez Ramos & Celia Rico Pérez
    2020 Study on the Usefulness of Machine Translation in the Migratory Context: Analysis of Translators’ Perceptions. Open Linguistics6 (1): 68–76. 10.1515/opli‑2020‑0004
    https://doi.org/10.1515/opli-2020-0004 [Google Scholar]
  49. Peris Abril, Álvaro, Miguel Domingo Ballester & Francisco Casacuberta Nolla
    2017 Interactive Neural Machine Translation. Computer Speech & Language451 (September): 201–20. 10.1016/j.csl.2016.12.003
    https://doi.org/10.1016/j.csl.2016.12.003 [Google Scholar]
  50. Sadiku, Matthew N. O. & Sarhan M. Musa
    2021A Primer on Multiple Intelligences. Cham: Springer International Publishing. 10.1007/978‑3‑030‑77584‑1
    https://doi.org/10.1007/978-3-030-77584-1 [Google Scholar]
  51. Sakamoto, Akiko
    2019 Unintended Consequences of Translation Technologies: From Project Managers Perspectives’. Perspectives27 (1): 58–73. 10.1080/0907676X.2018.1473452
    https://doi.org/10.1080/0907676X.2018.1473452 [Google Scholar]
  52. Sánchez-Torrón, Marina
    2017 Productivity in Post-Editing and in Neural Interactive Translation Prediction: A Study of English-to-Spanish Professional Translators. PhD Dissertation. University of Auckland. hdl.handle.net/2292/37205
  53. Sánchez Gijón, Pilar, Joss Moorkens & Andy Way
    2019 Post-Editing Neural Machine Translation versus Translation Memory Segments. Machine Translation33 (1–2): 31–59. 10.1007/s10590‑019‑09232‑x
    https://doi.org/10.1007/s10590-019-09232-x [Google Scholar]
  54. Sanchis Trilles, Germán, Vicent Alabau Gonzalvo, Christian Buck, Michael Carl, Francisco Casacuberta Nolla, Mercedes García Martínez, Ulrich Germann, Jesús González Rubio, Robin L. Hill, Philipp Koehn, Luis A. Leiva Torres, Bartolomé Mesa Lao, Daniel Ortiz Martínez, Herve Saint-Amand, Chara Tsoukala & Enrique Vidal Ruiz
    2014 Interactive Translation Prediction versus Conventional Post-Editing in Practice: A Study with the CasMaCat Workbench, Machine Translation28 (3): 217–235. 10.1007/s10590‑014‑9157‑9
    https://doi.org/10.1007/s10590-014-9157-9 [Google Scholar]
  55. Schrepp, Martin, Andreas Hinderks & Jörg Thomaschewski
    2014 Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios. InDesign, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, edited byAaron Marcus, 383–392. Lecture Notes in Computer Science. Cham: Springer. 10.1007/978‑3‑319‑07668‑3_37
    https://doi.org/10.1007/978-3-319-07668-3_37 [Google Scholar]
  56. Torregrosa Rivero, Daniel
    2018 Black-box interactive translation prediction. PhD dissertation, Universidad de Alicante. Http://purl.org/dc/dcmitype/Text
  57. Vermeeren, Arnold P. O. S., Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout & Kaisa Väänänen-Vainio-Mattila
    2010 User Experience Evaluation Methods: Current State and Development Needs. InProceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, 521–30. NordiCHI ’10. New York, NY, USA: Association for Computing Machinery. 10.1145/1868914.1868973
    https://doi.org/10.1145/1868914.1868973 [Google Scholar]
  58. Vieira, Lucas Nunes
    2019 Post-Editing of Machine Translation. InThe Routledge Handbook of Translation and Technology. Edited byMinako O’Hagan, 319–335. Routledge. 10.4324/9781315311258‑19
    https://doi.org/10.4324/9781315311258-19 [Google Scholar]
  59. Wang, Xiangling, Tingting Wang, Ricardo Muñoz Martín & Yanfang Jia
    2021 Investigating Usability in Postediting Neural Machine Translation: Evidence from Translation Trainees’ Self-Perception and Performance. Across Languages and Cultures22 (1): 100–123. 10.1556/084.2021.00006
    https://doi.org/10.1556/084.2021.00006 [Google Scholar]
  60. Weinberg, Bruce A.
    2004 ‘Experience and Technology Adoption’. SSRN Scholarly Paper. Rochester, NY. 10.2139/ssrn.522302
    https://doi.org/10.2139/ssrn.522302 [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error