1887
image of Missing the human touch?
USD
Buy:$35.00 + Taxes

Abstract

Abstract

Existing research suggests that machine translations of literary texts remain unsatisfactory. Such quality assessment often relies on automated metrics and subjective human ratings, with little attention to the stylistic features of machine translation. Current understanding is limited regarding the extent to which AI may transform the literary translation landscape, with implications for other critical domains for translation such as the creative industries more broadly. This pioneering study investigates the stylistic features of AI translations, specifically examining GPT-4’s performance against human translations of Chinese online literature. Our computational stylometry analysis reveals that GPT-4 translations closely mirror human translations in lexical, syntactic and content features. In addition to showing the relevance of stylometry for analysing the features of AI translation, the study provides critical insights into the implications of AI for literary translation in the posthuman tradition, where the line between machine and human translation becomes increasingly blurry.

Loading

Article metrics loading...

/content/journals/10.1075/ts.24043.yao
2025-11-24
2025-12-04
Loading full text...

Full text loading...

References

  1. Abbasi, Ahmed, and Hsinchun Chen
    2008 “Writeprints: A Stylometric Approach to Identity-Level Identification and Similarity Detection in Cyberspace.” ACM Transactions on Information Systems (): –. 10.1145/1361684.1361685
    https://doi.org/10.1145/1361684.1361685 [Google Scholar]
  2. Abu-Rayyash, Hussein
    2024 “AI Meets Comedy: Viewers’ Reactions to GPT-4 Generated Humor Translation.” Ampersand (): –. 10.1016/j.amper.2023.100162
    https://doi.org/10.1016/j.amper.2023.100162 [Google Scholar]
  3. Baker, Mona
    2000 “Towards a Methodology for Investigating the Style of a Literary Translator.” Target (): –. 10.1075/target.12.2.04bak
    https://doi.org/10.1075/target.12.2.04bak [Google Scholar]
  4. Burrows, John
    2002 “‘Delta’: A measure of stylistic difference and a guide to likely authorship.” Literary and Linguistic Computing (): –. 10.1093/llc/17.3.267
    https://doi.org/10.1093/llc/17.3.267 [Google Scholar]
  5. Cao, Qilin
    2021 “Accommodating Paratextual Theory to Born-Digital Literature Translation: A Case Study of ‘Wuxiaworld’.” Asia Pacific Translation and Intercultural Studies (): –. 10.1080/23306343.2021.1957346
    https://doi.org/10.1080/23306343.2021.1957346 [Google Scholar]
  6. Carl, Michael
    2021 “An Enactivist-Posthumanist Perspective on the Translation Process.” InContesting Epistemologies in Cognitive Translation and Interpreting Studies, edited bySandra L. Halverson and Álvaro Marín García, –. Routledge. 10.4324/9781003125792‑11
    https://doi.org/10.4324/9781003125792-11 [Google Scholar]
  7. Castilho, Sheila, Clodagh Mallon, Rahel Meister, and Shengya Yue
    2023 “Do Online Machine Translation Systems Care for Context? What About a GPT Model?” InProceedings of the 24th Annual Conference of the European Association for Machine Translation, –. Tampere, Finland.
    [Google Scholar]
  8. Castilho, Sheila, Stephen Doherty, Federico Gaspari, and Joss Moorkens
    2018 “Approaches to Human and Machine Translation Quality Assessment.” InTranslation Quality Assessment, edited byJoss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, –. Springer International Publishing. 10.1007/978‑3‑319‑91241‑7_2
    https://doi.org/10.1007/978-3-319-91241-7_2 [Google Scholar]
  9. Chang, Jie, and Gang Zhao
    2022 “The Reader”s Visibility: Analyzing Reader’s Intervention in Fan-Based Translation on Wuxiaworld.” Translation Review (): –. 10.1080/07374836.2022.2089414
    https://doi.org/10.1080/07374836.2022.2089414 [Google Scholar]
  10. Cipriani, Anna Maria
    2023Literary digital stylistics in translation studies. Springer. 10.1007/978‑981‑99‑6593‑9
    https://doi.org/10.1007/978-981-99-6593-9 [Google Scholar]
  11. Cope, Bill, and Mary Kalantzis
    2024 “A Multimodal Grammar of Artificial Intelligence: Measuring the Gains and Losses in Generative AI.” Multimodality & Society (): –. 10.1177/26349795231221699
    https://doi.org/10.1177/26349795231221699 [Google Scholar]
  12. Cronin, Michael
    2020 “Translation and Posthumanism.” InThe Routledge Handbook of Translation and Ethics, edited byKaisa Koskinen and Nike K. Pokorn, –. Routledge. 10.4324/9781003127970‑21
    https://doi.org/10.4324/9781003127970-21 [Google Scholar]
  13. Fang, Yu, and Haitao Liu
    2023 “Seeing Various Adventures through a Mirror: Detecting Translator’s Stylistic Visibility in Chinese Translations of Alice’s Adventure in Wonderland.” Digital Scholarship in the Humanities (): –. 10.1093/llc/fqac024
    https://doi.org/10.1093/llc/fqac024 [Google Scholar]
  14. Feng, Jin
    2013Romancing the internet: Producing and consuming Chinese web romance. Brill. 10.1163/9789004259720
    https://doi.org/10.1163/9789004259720 [Google Scholar]
  15. Fenoulhet, Jane
    2020 “The Relational Identity of the Translator.” Language and Intercultural Communication (): –. 10.1080/14708477.2020.1776722
    https://doi.org/10.1080/14708477.2020.1776722 [Google Scholar]
  16. Flynn, Peter
    2004 “Skopos Theory: An Ethnographic Enquiry.” Perspectives: Studies in Translatology (): –. 10.1080/0907676X.2004.9961507
    https://doi.org/10.1080/0907676X.2004.9961507 [Google Scholar]
  17. Gao, Yuan, Ruili Wang, and Feng Hou
    2023 “How to Design Translation Prompts for ChatGPT: An Empirical Study.” arXiv. 10.48550/arXiv.2304.02182
    https://doi.org/10.48550/arXiv.2304.02182 [Google Scholar]
  18. Gómez-Adorno, Helena, Juan-Pablo Posadas-Durán, Germán Ríos-Toledo, Grigori Sidorov, and Gerardo Sierra
    2018 “Stylometry-based Approach for Detecting Writing Style Changes in Literary Texts.” Computación y Sistemas (): –. 10.13053/cys‑22‑1‑2882
    https://doi.org/10.13053/cys-22-1-2882 [Google Scholar]
  19. Gourlay, Lesley
    2015 “Posthuman Texts: Nonhuman Actors, Mediators and the Digital University.” Social Semiotics (): –. 10.1080/10350330.2015.1059578
    https://doi.org/10.1080/10350330.2015.1059578 [Google Scholar]
  20. Guerberof-Arenas, Ana, and Antonio Toral
    2022 “Creativity in Translation: Machine Translation as a Constraint for Literary Texts.” Translation Spaces (): –. 10.1075/ts.21025.gue
    https://doi.org/10.1075/ts.21025.gue [Google Scholar]
  21. Haverals, Wouter, Lindsey Geybels, and Vanessa Joosen
    2022 “A Style for Every Age: A Stylometric Inquiry into Crosswriters for Children, Adolescents and Adults.” Language and Literature: International Journal of Stylistics (): –. 10.1177/09639470211072163
    https://doi.org/10.1177/09639470211072163 [Google Scholar]
  22. Hayles, N. Katherine
    1999How we became posthuman: Virtual bodies in cybernetics, literature, and informatics. University of Chicago Press. 10.7208/chicago/9780226321394.001.0001
    https://doi.org/10.7208/chicago/9780226321394.001.0001 [Google Scholar]
  23. Hendy, Amr, Mohamed Abdelrehim, Amr Sharaf,
    2023 “How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation.” arXiv. https://arxiv.org/abs/2302.09210
    [Google Scholar]
  24. Hockx, Michel
    2015Internet Literature in China. Columbia University Press.
    [Google Scholar]
  25. Huang, Libo
    2015Style in translation: A corpus-based perspective. Springer. 10.1007/978‑3‑662‑45566‑1
    https://doi.org/10.1007/978-3-662-45566-1 [Google Scholar]
  26. Ji, Meng, Pierrette Bouillon, and Mark Seligman
    2023Translation Technology in Accessible Health Communication. Cambridge University Press. 10.1017/9781108938976
    https://doi.org/10.1017/9781108938976 [Google Scholar]
  27. Jiang, Yuchen Eleanor, Tianyu Liu, Shuming Ma, Dongdong Zhang, Mrinmaya Sachan, and Ryan Cotterell
    2023 “Discourse centric evaluation of machine translation with a densely annotated parallel corpus.” arXiv. arxiv.org/abs/2305.11142
    [Google Scholar]
  28. Jiang, Yuchen, Tianyu Liu, Shuming Ma,
    2022 “BlonDe: An Automatic Evaluation Metric for Document-Level Machine Translation.” InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, –. Association for Computational Linguistics. 10.18653/v1/2022.naacl‑main.111
    https://doi.org/10.18653/v1/2022.naacl-main.111 [Google Scholar]
  29. Jiao, Wenxiang, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu
    2023 “Is ChatGPT a Good Translator? Yes with GPT-4 as the Engine.” arXiv. 10.48550/arXiv.2301.08745
    https://doi.org/10.48550/arXiv.2301.08745 [Google Scholar]
  30. Kang, Yong-Bin, Anthony McCosker, and Jane Farmer
    2023 “Leveraging Stylometric Analysis to Identify Unique Characteristics of Peer Support User Groups in Online Mental Health Forums.” Scientific Reports (): . 10.1038/s41598‑023‑50490‑w
    https://doi.org/10.1038/s41598-023-50490-w [Google Scholar]
  31. Kenny, Dorothy
    2020 “Machine Translation.” InRoutledge Encyclopedia of Translation Studies, 3rd ed., edited byMona Baker and Gabriela Saldanha, –. Routledge.
    [Google Scholar]
  32. 2025 “Literary Machine Translation: From Taboo to Controversy.” InThe Routledge Handbook of Translation Technology and Society, edited byStefan Baumgarten and Michael Tieber, –. Routledge. 10.4324/9781003271314‑33
    https://doi.org/10.4324/9781003271314-33 [Google Scholar]
  33. Kenny, Dorothy, and Marion Winters
    2020 “Machine Translation, Ethics and the Literary Translator’s Voice.” Translation Spaces (): –. 10.1075/ts.00024.ken
    https://doi.org/10.1075/ts.00024.ken [Google Scholar]
  34. 2024 “Customization, Personalization, and Style in Literary Machine Translation.” InTranslation, Interpreting and Technological Change: Innovations in Research, Practice and Training, edited byMarion Winters, Sharon Deane-Cox, and Ursula Böser, –. Bloomsbury. 10.5040/9781350212978.0010
    https://doi.org/10.5040/9781350212978.0010 [Google Scholar]
  35. Krüger, Ralph
    2022 “Some Translation Studies Informed Suggestions for Further Balancing Methodologies for Machine Translation Quality Evaluation.” Translation Spaces (): –. 10.1075/ts.21026.kru
    https://doi.org/10.1075/ts.21026.kru [Google Scholar]
  36. Kumarage, Tharindu, Joshua Garland, Amrita Bhattacharjee, Scott Ruston, and Huan Liu
    2023 “Stylometric Detection of AI-Generated Text in Twitter Timelines.” arXiv. 10.48550/arXiv.2303.03697
    https://doi.org/10.48550/arXiv.2303.03697 [Google Scholar]
  37. Lagutina, Ksenia, Nadezhda Lagutina, Elena Boychuk, Inna Vorontsova, Elena Shliakhtina, Olga Belyaeva, Ilya Paramonov, and P. G. Demidov
    2019 “A Survey on Stylometric Text Features.” In2019 25th Conference of Open Innovations Association (FRUCT), –. IEEE. 10.23919/FRUCT48121.2019.8981504
    https://doi.org/10.23919/FRUCT48121.2019.8981504 [Google Scholar]
  38. Lee, Changsoo
    2022 “How Do Machine Translators Measure up to Human Literary Translators in Stylometric Tests?” Digital Scholarship in the Humanities (): –. 10.1093/llc/fqab091
    https://doi.org/10.1093/llc/fqab091 [Google Scholar]
  39. Lee, Tong King
    2023 “Artificial Intelligence and Posthumanist Translation: ChatGPT versus the Translator.” Applied Linguistics Review. 10.1515/applirev‑2023‑0122
    https://doi.org/10.1515/applirev-2023-0122 [Google Scholar]
  40. Li, Dang
    2021 “The Transcultural Flow and Consumption of Online Wuxia Literature through Fan-Based Translation.” Interventions (): –. 10.1080/1369801X.2020.1854815
    https://doi.org/10.1080/1369801X.2020.1854815 [Google Scholar]
  41. Li, Defeng, Chunling Zhang, and Kanglong Liu
    2011 “Translation Style and Ideology: A Corpus-Assisted Analysis of Two English Translations of Hongloumeng.” Literary and Linguistic Computing (): –. 10.1093/llc/fqr001
    https://doi.org/10.1093/llc/fqr001 [Google Scholar]
  42. Moorkens, Joss, Antonio Toral, Sheila Castilho, and Andy Way
    2018 “Translators’ Perceptions of Literary Post-Editing Using Statistical and Neural Machine Translation.” Translation Spaces (): –. 10.1075/ts.18014.moo
    https://doi.org/10.1075/ts.18014.moo [Google Scholar]
  43. O’Brien, Sharon
    2020 “Translation, human–computer interaction and cognition 1.” InThe Routledge handbook of translation and cognition, edited byFabio Alves and Arnt Jakobsen, –. Routledge. 10.4324/9781315178127‑25
    https://doi.org/10.4324/9781315178127-25 [Google Scholar]
  44. 2024 “Human-Centered Augmented Translation: Against Antagonistic Dualisms.” Perspectives (): –. 10.1080/0907676X.2023.2247423
    https://doi.org/10.1080/0907676X.2023.2247423 [Google Scholar]
  45. O’Thomas, Mark
    2017 “Humanum Ex Machina: Translation in the Post-Global, Posthuman World.” Target. International Journal of Translation Studies (): –. 10.1075/target.29.2.05oth
    https://doi.org/10.1075/target.29.2.05oth [Google Scholar]
  46. Omar, Abdulfattah, and Yasser Gomaa
    2020 “The Machine Translation of Literature: Implications for Translation Pedagogy.” International Journal of Emerging Technologies in Learning (): –. 10.3991/ijet.v15i11.13275
    https://doi.org/10.3991/ijet.v15i11.13275 [Google Scholar]
  47. Ouyang, Youquan
    2023A history of cyber literary criticism in China. Routledge.
    [Google Scholar]
  48. Peng, Keqin, Liang Ding, Qihuang Zhong,
    2023 “Towards Making the Most of ChatGPT for Machine Translation.” SSRN Electronic Journal. 10.2139/ssrn.4390455
    https://doi.org/10.2139/ssrn.4390455 [Google Scholar]
  49. Ping, Yuan, and Binhua Wang
    2024 “Retranslated Chinese Classical Canon Journey to the West: A Stylometric Comparison between Julia Lovell’s Retranslation and Arthur Waley’s Translation.” Digital Scholarship in the Humanities (): –. 10.1093/llc/fqad085
    https://doi.org/10.1093/llc/fqad085 [Google Scholar]
  50. Ragni, Valentina, and Lucas Nunes Vieira
    2022 “What Has Changed with Neural Machine Translation? A Critical Review of Human Factors.” Perspectives (): –. 10.1080/0907676X.2021.1889005
    https://doi.org/10.1080/0907676X.2021.1889005 [Google Scholar]
  51. Raunak, Vikas, Arul Menezes, Matt Post, and Hany Hassan
    2023 “Do GPTs Produce Less Literal Translations?” InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), –. Association for Computational Linguistics. 10.18653/v1/2023.acl‑short.90
    https://doi.org/10.18653/v1/2023.acl-short.90 [Google Scholar]
  52. Rybicki, Jan, and Magda Heydel
    2013 “The Stylistics and Stylometry of Collaborative Translation: Woolf’s Night and Day in Polish.” Literary and Linguistic Computing (): –. 10.1093/llc/fqt027
    https://doi.org/10.1093/llc/fqt027 [Google Scholar]
  53. Rybicki, Jan
    2006 “Burrowing into Translation: Character Idiolects in Henryk Sienkiewicz’s Trilogy and Its Two English Translations.” Literary and Linguistic Computing (): –. 10.1093/llc/fqh051
    https://doi.org/10.1093/llc/fqh051 [Google Scholar]
  54. 2012 “The Great Mystery of the (Almost) Invisible Translator: Stylometry in Translation.” InQuantitative Methods in Corpus-Based Translation Studies, edited byMichael Oakes and Meng Ji, –. John Benjamins. 10.1075/scl.51.09ryb
    https://doi.org/10.1075/scl.51.09ryb [Google Scholar]
  55. 2025 “Can Machine Translation of Literary Texts Fool Stylometry?” Digital Scholarship in the Humanities (): –. 10.1093/llc/fqaf010
    https://doi.org/10.1093/llc/fqaf010 [Google Scholar]
  56. Ryu, Jisu, Soonbae Kim, Arthur C. Graesser, and Moongee Jeon
    2023 “Corpus Stylistic Analysis of Literary Translation Using Multilevel Linguistic Measures: James Joyce’s Dubliners and A Portrait of the Artist as a Young Man and Their Korean Translations.” Target (): –. 10.1075/target.21131.ryu
    https://doi.org/10.1075/target.21131.ryu [Google Scholar]
  57. Shao, Yanjun
    2015 Wangluo wenxue de wangluoxing yu jingdianxing [The internetization and classicalization of Internet literature]. Journal of Peking University (Philosophy and Social Sciences)(): –.
    [Google Scholar]
  58. Siu, Sai Cheong
    2023 “ChatGPT and GPT-4 for Professional Translators: Exploring the Potential of Large Language Models in Translation.” SSRN. 10.2139/ssrn.4448091
    https://doi.org/10.2139/ssrn.4448091 [Google Scholar]
  59. Sørbø, Marie N.
    2018Jane Austen speaks Norwegian: The challenges of literary translation. Brill. 10.1163/9789004337176
    https://doi.org/10.1163/9789004337176 [Google Scholar]
  60. Sun, Yifeng, and Kanglong Liu
    2025 “Stylistic Nuances through Syntactic Complexity: A Corpus-Assisted Study of Narration and Dialogue in Two English Translations of Hongloumeng.” Applied Corpus Linguistics (): . 10.1016/j.acorp.2025.100125
    https://doi.org/10.1016/j.acorp.2025.100125 [Google Scholar]
  61. Thai, Katherine, Marzena Karpinska, Kalpesh Krishna,
    2022 “Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature.” arXiv. arxiv.org/abs/2210.14250. 10.18653/v1/2022.emnlp‑main.672
    https://doi.org/10.18653/v1/2022.emnlp-main.672 [Google Scholar]
  62. Tian, Xiaoli, and Michael Adorjan
    2016 “Fandom and Coercive Empowerment: The Commissioned Production of Chinese Online Literature.” Media, Culture & Society (): –. 10.1177/0163443716646172
    https://doi.org/10.1177/0163443716646172 [Google Scholar]
  63. Toral, Antonio, and Andy Way
    2015 “Machine-Assisted Translation of Literary Text: A Case Study.” Translation Spaces (): –. 10.1075/ts.4.2.04tor
    https://doi.org/10.1075/ts.4.2.04tor [Google Scholar]
  64. 2018 “What Level of Quality Can Neural Machine Translation Attain on Literary Text?” InTranslation Quality Assessment: From Principles to Practice, edited byJoss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, –. Springer. 10.1007/978‑3‑319‑91241‑7_12
    https://doi.org/10.1007/978-3-319-91241-7_12 [Google Scholar]
  65. Vinay, Jean-Paul, and Jean Darbelnet
    1995Comparative Stylistics of French and English: A Methodology for Translation. Translated byJuan C. Sager and Marie-Jo Hamel. John Benjamins. 10.1075/btl.11
    https://doi.org/10.1075/btl.11 [Google Scholar]
  66. Winters, Marion, and Dorothy Kenny
    2023 “Mark My Keywords: A Translator-Specific Exploration of Style in Literary Machine Translation.” InComputer-Assisted Literary Translation, edited byAndrew Rothwell, Andy Way, and Roy Youdale, –. Routledge. 10.4324/9781003357391‑5
    https://doi.org/10.4324/9781003357391-5 [Google Scholar]
  67. Wu, Kan, and Dechao Li
    2022 “Are Translated Chinese Wuxia Fiction and Western Heroic Literature Similar? A Stylometric Analysis Based on Stylistic Panoramas.” Digital Scholarship in the Humanities (): –. 10.1093/llc/fqac019
    https://doi.org/10.1093/llc/fqac019 [Google Scholar]
  68. Wu, Min-Hua
    2024 “Representing the Poetic Styles of Hamlet in Equivalent Chinese: Perng Ching-His’s Literary Translation of Shakespeare.” The Translator (): –. 10.1080/13556509.2023.2299065
    https://doi.org/10.1080/13556509.2023.2299065 [Google Scholar]
  69. Yamada, Masaru
    2023 “Optimizing Machine Translation through Prompt Engineering: An Investigation into ChatGPT’s Customizability.” InProceedings of Machine Translation Summit XIX, Vol. 2: Users Track, –. 10.48550/arXiv.2308.01391
    https://doi.org/10.48550/arXiv.2308.01391 [Google Scholar]
  70. Zaitsu, Wataru, and Mingzhe Jin
    2023 “Distinguishing ChatGPT(-3.5, -4)-Generated and Human-Written Papers through Japanese Stylometric Analysis.” PLOS ONE (): –. 10.1371/journal.pone.0288453
    https://doi.org/10.1371/journal.pone.0288453 [Google Scholar]
  71. Zhang, Biao, Barry Haddow, and Alexandra Birch
    2023 “Prompting Large Language Model for Machine Translation: A Case Study.” Proceedings of the 40th International Conference on Machine Learning, PMLR: –. https://proceedings.mlr.press/v202/zhang23m.html
    [Google Scholar]
/content/journals/10.1075/ts.24043.yao
Loading
/content/journals/10.1075/ts.24043.yao
Loading

Data & Media loading...

  • Article Type: Research Article
Keywords: stylometry ; large language models ; GPT ; translation ; posthumanism
This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error