Volume 34, Issue 2
  • ISSN 0924-1884
  • E-ISSN: 1569-9986
Buy:$35.00 + Taxes



Widely used computer-aided translation (CAT) tools divide documents into segments, such as sentences, and arrange them side-by-side in a spreadsheet-like view. We present the first controlled evaluation of these design choices on translator performance, measuring speed and accuracy in three experimental text-processing tasks. We find significant evidence that sentence-by-sentence presentation enables faster text reproduction and within-sentence error identification compared to unsegmented text, and that a top-and-bottom arrangement of source and target sentences enables faster text reproduction compared to a side-by-side arrangement. For revision, on the other hand, we find that presenting unsegmented text results in the highest accuracy and time efficiency. Our findings have direct implications for best practices in designing CAT tools.


Article metrics loading...

Loading full text...

Full text loading...


  1. Arif, Ahmed Sabbir, and Wolfgang Stuerzlinger
    2009 “Analysis of Text Entry Performance Metrics.” In2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC–STH), 26–27 September 2009, Toronto Canada, 100–105. Piscataway, NJ: IEEE. 10.1109/TIC‑STH.2009.5444533
    https://doi.org/10.1109/TIC-STH.2009.5444533 [Google Scholar]
  2. Baayen, R. Harald, and Petar Milin
    2010 “Analyzing Reaction Times.” International Journal of Psychological Research3 (2): 12–28. 10.21500/20112084.807
    https://doi.org/10.21500/20112084.807 [Google Scholar]
  3. Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker
    2015 “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software67 (1): 1–48. 10.18637/jss.v067.i01
    https://doi.org/10.18637/jss.v067.i01 [Google Scholar]
  4. Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck
    2017 “Findings of the 2017 Conference on Machine Translation (WMT17).” InWMT 2017: Second Conference on Machine Translation: Proceedings, edited byOndřej Bojar, Christian Buck, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck , 169–214. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/W17‑4717
    https://doi.org/10.18653/v1/W17-4717 [Google Scholar]
  5. Bojar, Ondřej, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn
    2018 “Findings of the 2018 Conference on Machine Translation (WMT18).” InWMT 2018: Third Conference on Machine Translation: Proceedings of the Conference, edited byOndřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck , 272–307. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/W18‑6401
    https://doi.org/10.18653/v1/W18-6401 [Google Scholar]
  6. Bowker, Lynne
    2005 “Productivity vs Quality? A Pilot Study on the Impact of Translation Memory Systems.” Localisation Focus4 (1): 13–20.
    [Google Scholar]
  7. Campbell, Stuart
    1999 “A Cognitive Approach to Source Text Difficulty in Translation.” Target11 (1): 33–63. 10.1075/target.11.1.03cam
    https://doi.org/10.1075/target.11.1.03cam [Google Scholar]
  8. Carl, Michael
    2010 “A Computational Framework for a Cognitive Model of Human Translation Processes.” InProceedings of Translating and the Computer 32. London: Aslib.
    [Google Scholar]
  9. 2012 “Translog-II: A Program for Recording User Activity Data for Empirical Reading and Writing Research.” InProceedings of the Eighth International Conference on Language Resources and Evaluation (LREC ’12), edited byNicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno , 4108–4112. Istanbul: European Language Resources Association.
    [Google Scholar]
  10. Castilho, Sheila, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way
    2017 “Is Neural Machine Translation the New State of the Art?” The Prague Bulletin of Mathematical Linguistics108: 109–120. 10.1515/pralin‑2017‑0013
    https://doi.org/10.1515/pralin-2017-0013 [Google Scholar]
  11. Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Vilelmini Sosoni, Panayota Georgakopoulou, Pintu Lohar
    2017 “A Comparative Quality Evaluation of PBSMT and NMT Using Professional Translators.” InProceedings of Machine Translation Summit XVI, vol.1, edited bySadao Kurohashi and Pascale Fung, 116–131. Nagoya: Asia-Pacific Association for Machine Translation.
    [Google Scholar]
  12. Castilho, Sheila, Joss Moorkens, Federico Gaspari, Rico Sennrich, Andy Way, and Panayota Georgakopoulou
    2018 “Evaluating MT for Massive Open Online Courses.” InHuman Evaluation of Statistical and Neural Machine Translation, edited byAndy Way and Mikel L. Forcada, special issue ofMachine Translation32 (3): 255–278. 10.1007/s10590‑018‑9221‑y
    https://doi.org/10.1007/s10590-018-9221-y [Google Scholar]
  13. Coppers, Sven, Jan van den Bergh, Kris Luyten, Karin Coninx, Iulianna van der Lek–Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste
    2018 “Intellingo: An Intelligible Translation Environment.” InProceedings of the 2018 CHI Conference on Human Factors in Computing Systems, edited byRegan Mandryk, Mark Hancock, Mark Perry, and Anna Cox. Montreal: Association for Computing Machinery. 10.1145/3173574.3174098
    https://doi.org/10.1145/3173574.3174098 [Google Scholar]
  14. do Carmo, Félix, and Joss Moorkens
    2020 “Differentiating Editing, Post-Editing and Revision.” InTranslation Revision and Post-Editing: Industry Practices and Cognitive Processes, edited byMaarit Koponen, Brian Mossop, Isabelle S. Robert, and Giovanna Scocchera. Abingdon: Routledge. 10.4324/9781003096962‑4
    https://doi.org/10.4324/9781003096962-4 [Google Scholar]
  15. Dragsted, Barbara
    2006 “Computer-Aided Translation as a Distributed Cognitive Task.” InDistributed Cognition, edited byStevan Harnad and Itiel E. Dror, special issue ofPragmatics & Cognition14 (2): 443–464. 10.1075/pc.14.2.17dra
    https://doi.org/10.1075/pc.14.2.17dra [Google Scholar]
  16. 2010 “Coordination of Reading and Writing Processes in Translation: An Eye on Uncharted Territory.” InTranslation and Cognition, edited byGregory M. Shreve and Erik Angelone, 41–62. Amsterdam: John Benjamins. 10.1075/ata.xv.04dra
    https://doi.org/10.1075/ata.xv.04dra [Google Scholar]
  17. Ehrensberger-Dow, Maureen, Andrea Hunziker Heeb, Gary Massey, Ursula Meidert, Silke Neumann, and Heidrun Becker
    2016 “An International Survey of the Ergonomics of Professional Translation.” InErgonomic Approaches to Professional Practices and Translator Training, edited byÉlisabeth Lavault-Olléon, special issue ofJournal of the Institute of Languages and Cultures of Europe, America, Africa, Asia and Australia27. 10.4000/ilcea.4004
    https://doi.org/10.4000/ilcea.4004 [Google Scholar]
  18. Engelbart, Douglas C., and William K. English
    1968 “A Research Center for Augmenting Human Intellect.” InAFIPS 1968 Fall Joint Conference, Part 1, 395–410. Washington: Thompson Book Company. 10.1145/1476589.1476645
    https://doi.org/10.1145/1476589.1476645 [Google Scholar]
  19. Ericsson, K. Anders, and Herbert A. Simon
    1984Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.
    [Google Scholar]
  20. Federico, Marcello, Alessandro Cattelan, and Marco Trombetti
    2012 “Measuring User Productivity in Machine Translation Enhanced Computer Assisted Translation.” InProceedings of the 10th Conference of the Association for Machine Translation in the Americas. San Diego: Association for Machine Translation in the Americas.
    [Google Scholar]
  21. Francis, Wendy S., and Silvia P. Sáenz
    2007 “Repetition Priming Endurance in Picture Naming and Translation: Contributions of Component Processes.” Memory & Cognition35 (3): 481–493. 10.3758/BF03193288
    https://doi.org/10.3758/BF03193288 [Google Scholar]
  22. Gelman, Andrew, and Jennifer Hill
    2007Data Analysis Using Regression and Multilevel/Hierarchical Models. New York: Cambridge University Press.
    [Google Scholar]
  23. Green, Spence, Jason Chuang, Jeffrey Heer, and Christopher D. Manning
    2014 “Predictive Translation Memory: A Mixed-Initiative System for Human Language Translation.” InUIST ’14: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, edited byHrvoje Benko, Mira Dontcheva, and Daniel Wigdor, 177–187. New York: Association for Computing Machinery. 10.1145/2642918.2647408
    https://doi.org/10.1145/2642918.2647408 [Google Scholar]
  24. Hassan, Hany, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang
    2018 “Achieving Human Parity on Automatic Chinese to English News Translation.” arXiv Preprint1803.05567. 10.29007/sbcv
    https://doi.org/10.29007/sbcv [Google Scholar]
  25. Hornbæk, Kasper, and Erik Frøkjær
    2001 “Reading of Electronic Documents: The Usability of Linear, Fisheye, and Overview+detail Interfaces.” InCHI ’01: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, edited byJulie Jacko and Andrew Sears, 293–300. New York: Association for Computing Machinery. 10.1145/365024.365118
    https://doi.org/10.1145/365024.365118 [Google Scholar]
  26. House, Juliane
    2013 “Quality in Translation Studies.” InThe Routledge Handbook of Translation, edited byCarmen Millán and Francesca Bartrina, 534–547. Abingdon: Routledge.
    [Google Scholar]
  27. Jakobsen, Arnt Lykke
    2003 “Effects of Think Aloud on Translation Speed, Revision, and Segmentation.” InTriangulating Translation: Perspectives in Process Oriented Research, edited byFabio Alves, 69–95. Amsterdam: John Benjamins. 10.1075/btl.45.08jak
    https://doi.org/10.1075/btl.45.08jak [Google Scholar]
  28. Junczys-Dowmunt, Marcin
    2019 “Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation.” InProceedings of the Fourth Conference on Machine Translation (WMT19), vol.2, edited byLoïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow , 225–233. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/W19‑5321
    https://doi.org/10.18653/v1/W19-5321 [Google Scholar]
  29. Karimova, Sariya, Patrick Simianer, and Stefan Riezler
    2018 “A User-Study on Online Adaptation of Neural Machine Translation to Human Post-Edits.” Machine Translation32 (4): 309–324. 10.1007/s10590‑018‑9224‑8
    https://doi.org/10.1007/s10590-018-9224-8 [Google Scholar]
  30. Kay, Martin
    1980 “The Proper Place of Men and Machines in Language Translation.” Research Report CSL-80-11. Palo Alto, CA: Xerox Palo Alto Research Center.
    [Google Scholar]
  31. Krings, Hans P.
    1994Texte Reparieren: Empirische Untersuchungen Zum Prozeß Der Nachredaktion von Maschinenübersetzungen. Habilitation thesis. University of Hildesheim.
    [Google Scholar]
  32. 2001Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes. Translated byGeoffrey S. Koby, Gregory M. Shreve, Katja Mischerikow, and Sarah Litzer. Kent: Kent State University Press.
    [Google Scholar]
  33. Langlais, Philippe, Guy Lapalme, and Sébastien Sauvé
    2001 “User Interface Aspects of a Translation Typing System.” InAdvances in Artificial Intelligence: 14th Biennial Conference of the Canadian Society for Computational Studies of Intelligence, AI 2001, edited byEleni Stroulia and Stan Matwin, 246–256. Berlin: Springer. 10.1007/3‑540‑45153‑6_24
    https://doi.org/10.1007/3-540-45153-6_24 [Google Scholar]
  34. Läubli, Samuel, Rico Sennrich, and Martin Volk
    2018 “Has Machine Translation Achieved Human Parity? A Case for Document-Level Evaluation.” InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, edited byEllen Riloff, Davide Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, 4791–4796. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/D18‑1512
    https://doi.org/10.18653/v1/D18-1512 [Google Scholar]
  35. LeBlanc, Matthieu
    2013 “Translators on Translation Memory (TM): Results of an Ethnographic Study in Three Translation Services and Agencies.” Translation & Interpreting5 (2): 1–13. 10.12807/ti.105202.2013.a01
    https://doi.org/10.12807/ti.105202.2013.a01 [Google Scholar]
  36. Macklovitch, Elliott
    2006 “TransType2: The Last Word.” InProceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), edited byNicoletta Calzolari, Khalid Choukri, Aldo Gangemi, Bente Maegaard, Joseph Mariani, Jan Odijk, and Daniel Tapias, 167–172. European Language Resources Association.
    [Google Scholar]
  37. Green, Spence, Jeffrey Heer, and Christopher D. Manning
    2013 “The Efficacy of Human Post-Editing for Language Translation.” InCHI 2013: Changing Perspectives, Conference Proceedings, edited byWendy E. Mackay, Stephen Brewster, and Susanne Bødker. New York: Association for Computing Machinery. 10.1145/2470654.2470718
    https://doi.org/10.1145/2470654.2470718 [Google Scholar]
  38. Miniukovich, Aliaksei, Antonella de Angeli, Simone Sulpizio, and Paola Venuti
    2017 “Design Guidelines for Web Readability.” InDIS ’17: Proceedings of the 2017 Conference on Designing Interactive Systems, edited byOli Mival, Michael Smyth, and Peter Dalsgaard, 285–296. New York: Association for Computing Machinery. 10.1145/3064663.3064711
    https://doi.org/10.1145/3064663.3064711 [Google Scholar]
  39. Moorkens, Joss, and Sharon O’Brien
    2017 “Assessing User Interface Needs of Post-Editors of Machine Translation.” InHuman Issues in Translation Technology, edited byDorothy Kenny, 109–130. Abingdon: Routledge.
    [Google Scholar]
  40. Müller, Mathias, Annette Rios, Elena Voita, and Rico Sennrich
    2018 “A Large-Scale Test Set for the Evaluation of Context-Aware Pronoun Translation in Neural Machine Translation.” InWMT 2018: Third Conference on Machine Translation: Proceedings of the Conference, edited byOndřej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck , 61–72. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/W18‑6307
    https://doi.org/10.18653/v1/W18-6307 [Google Scholar]
  41. Norman, Donald
    1988The Psychology of Everyday Things. New York: Basic Books.
    [Google Scholar]
  42. O’Brien, Sharon
    2009 “Eye Tracking in Translation Process Research: Methodological Challenges and Solutions.” InMethodology, Technology and Innovation in Translation Process Research: A Tribute to Arnt Lykke Jakobsen, edited byInger M. Mees, Fabio Alves, and Susanne Göpferich, 251–266. Frederiksberg: Samfundslitteratur.
    [Google Scholar]
  43. O’Brien, Sharon, Maureen Ehrensberger-Dow, Megan Connolly, and Marcel Hasler
    2017 “Irritating CAT Tool Features That Matter to Translators.” HERMES56: 145–162.
    [Google Scholar]
  44. Rello, Luz, Martin Pielot, and Mari-Carmen Marcos
    2016 “Make It Big!: The Effect of Font Size and Line Spacing on Online Readability.” InProceedings: The 32th Annual CHI Conference on Human Factors in Computing Science, edited byJofish Kaye, Allison Druin, Cliff Lampe, Dan Morris, Juan Pablo Hourcade, 3637–3648. New York: Association for Computing Machinery.
    [Google Scholar]
  45. Roberts, Teresa L.
    1980Evaluation of Computer Text Editors. PhD diss.Stanford University.
    [Google Scholar]
  46. Roberts, Teresa L., and Thomas P. Moran
    1982 “Evaluation of Computer Text Editors.” InCHI ’82: Proceedings of the 1982 Conference on Human Factors in Computing Systems, edited byJean A. Nichols and Michael L. Schneider, 136–141. New York: Association for Computing Machinery. 10.1145/800049.801770
    https://doi.org/10.1145/800049.801770 [Google Scholar]
  47. Ruiz, Carmen, Natalia Paredes, Pedro Macizo, and Maria Teresa Bajo
    2008 “Activation of Lexical and Syntactic Target Language Properties in Translation.” Acta Psychologica128 (3): 490–500. 10.1016/j.actpsy.2007.08.004
    https://doi.org/10.1016/j.actpsy.2007.08.004 [Google Scholar]
  48. Saffer, Dan
    2005The Role of Metaphor in Interaction Design. Master’s thesis. Carnegie Mellon University.
    [Google Scholar]
  49. Schneider, Dominik, Marcos Zampieri, and Josef van Genabith
    2018 “Translation Memories and the Translator: A Report on a User Survey.” Babel64 (5/6): 734–762. 10.1075/babel.00062.sch
    https://doi.org/10.1075/babel.00062.sch [Google Scholar]
  50. Sennrich, Rico
    2017 “How Grammatical Is Character-Level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs.” In15th Conference of the European Chapter of the Association for Computational Linguistics: Proceedings of the Conference, vol.2, edited byMirella Lapata, Phil Blunsom, and Alexander Koller, 376–382. Stroudsburg: Association for Computational Linguistics. 10.18653/v1/E17‑2060
    https://doi.org/10.18653/v1/E17-2060 [Google Scholar]
  51. Shih, Claire Yi-yi
    2006 “Revision from Translators’ Point of View: An Interview Study.” Target18 (2): 295–312. 10.1075/target.18.2.05shi
    https://doi.org/10.1075/target.18.2.05shi [Google Scholar]
  52. Shneiderman, Ben
    1983 “Direct Manipulation: A Step Beyond Programming Languages.” Computer8 (16): 57–69. 10.1109/MC.1983.1654471
    https://doi.org/10.1109/MC.1983.1654471 [Google Scholar]
  53. Yu, Chen-Hsiang, and Robert C. Miller
    2010 “Enhancing Web Page Readability for Non-Native Readers.” InCHI ’10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, edited byElizabeth Mynatt, Geraldine Fitzpatrick, Scott Hudson, Keith Edwards, and Tom Rodden, 2523–2532. New York: Association for Computing Machinery. 10.1145/1753326.1753709
    https://doi.org/10.1145/1753326.1753709 [Google Scholar]
  54. Zaretskaya, Anna
    2015D2.1: User Requirement Analysis. Technical Report: EXPERT: EXPloiting Empirical appRoaches to Translation, the European Union’s Seventh Framework Programme (FP7). expert-itn.eu/sites/default/files/outputs/expert_d2.1_20150210.pdf
    [Google Scholar]

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error