1887
Volume 21, Issue 2
  • ISSN 1598-7647
  • E-ISSN: 2451-909X
USD
Buy:$35.00 + Taxes

Abstract

Abstract

This paper aims to investigate the quality Korean–English patent translations by three machine translation (MT) engines based on automatic and human evaluations of Korean to English Patent Automatic Translation (K2E-PAT), a pattern-based statistical MT; and Patent Translate and WIPO Translate, both neural MTs. For title translations, WIPO Translate scored the highest in automatic and human evaluations, while results were mixed for the other two MTs. K2E-PAT slightly outperformed Patent Translate in automatic evaluation, whereas Patent Translate outperformed K2E-PAT in human evaluation. For abstract translations, Patent Translate scored the highest in automatic evaluation, followed by WIPO Translate and K2E-PAT. In human evaluation, the ranking order was the same as that of title translations, with WIPO Translate scoring the highest on average. The results indicated correlations between automatic and human evaluations, and the NMTs subject to the current study still do not render satisfactory gist translation from Korean to English.

Loading

Article metrics loading...

/content/journals/10.1075/forum.00030.lee
2023-11-14
2024-10-07
Loading full text...

Full text loading...

References

  1. Adams, Stephen
    2020Information Sources in Patents. Berlin: Walter de Gruyter GmbH & Co KG. 10.1515/9783110552263
    https://doi.org/10.1515/9783110552263 [Google Scholar]
  2. Banerjee, Satanjeev, and Alon Lavie
    2005 “METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments.” InProceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, Ann Arbor, Michigan, 29 June, ed. byJade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss, 65–72. New Jersey: The Association for Computational Linguistics.
    [Google Scholar]
  3. Bazrafshan, Marzieh
    2014 Semantic Features for Statistical Machine Translation. PhD diss.University of Rochester.
  4. Brkić, Marija, Sanja Seljan, and Tomislav Vičić
    2013 “Automatic and Human Evaluation on English-Croatian Legislative Test Set.” InComputational Linguistics and Intelligent Text Processing. ed. byAlexander Gelbukh, 311–317. Berlin, Heidelberg: Springer. 10.1007/978‑3‑642‑37256‑8_26
    https://doi.org/10.1007/978-3-642-37256-8_26 [Google Scholar]
  5. Callison-Burch, Chris, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder
    2007 “(Meta-)evaluation of Machine Translation.” InProceedings of the Second Workshop on Statistical Machine Translation, Prague, Czech Republic, 23 June, ed. byChris Callison-Burch, Philipp Koehn, Christof Monz, and Cameron Shaw Fordyce, 136–158. Pennsylvania: Association for Computational Linguistics. 10.3115/1626355.1626373
    https://doi.org/10.3115/1626355.1626373 [Google Scholar]
  6. Castilho, Sheila,
    2017 “Is Neural Machine Translation the New State of the Art?” The Prague Bulletin of Mathematical Linguistics1081: 109–120. 10.1515/pralin‑2017‑0013
    https://doi.org/10.1515/pralin-2017-0013 [Google Scholar]
  7. 2018 “Approaches to Human and Machine Translation Quality Assessment.” InTranslation Quality Assessment, ed. byJoss Moorkens, Sheila Castilho, Federico Gaspari and Stephen Doherty, 9–38. Cham: Springer. 10.1007/978‑3‑319‑91241‑7_2
    https://doi.org/10.1007/978-3-319-91241-7_2 [Google Scholar]
  8. Chatzikoumi, Eirini
    2020 “How to Evaluate Machine Translation: A Review of Automated and Human Metrics.” Natural Language Engineering261: 137–161. 10.1017/S1351324919000469
    https://doi.org/10.1017/S1351324919000469 [Google Scholar]
  9. Choi, Hyoeun, and Jieun Lee
    2017 “A Study on the Evaluation of Korean-English Patent Machine Translation–Focusing on KIPRIS K2E-PAT Translation.” Interpretation and Translation19 (1): 139–178. 10.20305/it201701139178
    https://doi.org/10.20305/it201701139178 [Google Scholar]
  10. Choi, Hyoeun, Chung-ho Lee, and Jun-ho Lee
    2023 “Evaluation of English-Korean Patent Machine Translations by a Patent-Specific NMT Engine Using AutoML.” Journal of Translation Studies24(2): 101–130.
    [Google Scholar]
  11. Choi, YooChan
    2009 “Korean to English Patent Automatic Translation (K2E-PAT) and Cross-Lingual Retrieval on KIPRIS.” World Patent Information311: 135–136. 10.1016/j.wpi.2008.09.005
    https://doi.org/10.1016/j.wpi.2008.09.005 [Google Scholar]
  12. Coughlin, Deborah
    2003 “Correlating Automated and Human Assessments of Machine Translation Quality.” Paper presented atMT Summit IX, New Orleans, US.
    [Google Scholar]
  13. Encyclopaedia Britannica
    Encyclopaedia Britannica 2022 “Electrostriction.” AccessedApril 4, 2022. https://www.britannica.com/science/electrostriction
  14. European Patent Office
    European Patent Office 2021 “Patent Translate.” AccessedFebruary 17, 2022. https://www.epo.org/searching-for-patents/helpful-resources/patent-translate.html
  15. Forcada, Mikel L.
    2017 “Making Sense of Neural Machine Translation.” Translation Spaces6 (2): 291–309. 10.1075/ts.6.2.06for
    https://doi.org/10.1075/ts.6.2.06for [Google Scholar]
  16. Kinoshita, Satoshi, Tadaaki Oshio, and Tomoharu Mitsuhashi
    2017 “Comparison of SMT and NMT trained with large Patent Corpora: Japio at WAT2017.” InProceedings of the 4th Workshop on Asian Translation, Taipei, Taiwan, 27 November–1 December, ed. byToshiaki Nakazawa , 140–145. Singapore: Asian Federation of Natural Language Processing.
    [Google Scholar]
  17. Koehn, Philipp, and Rebecca Knowles
    2017 “Six Challenges for Neural Machine Translation.” InProceedings of the First Workshop on Neural Machine Translation, Vancouver, Canada, 4 August, ed. byMinh-Thang Luong, Alexandra Birch, Grahan Neubig, and Andrew Finch, 28–39. Pennsylvania: Association for Computational Linguistics. 10.18653/v1/W17‑3204
    https://doi.org/10.18653/v1/W17-3204 [Google Scholar]
  18. Koehn, Philipp
    2007 “Moses: Open-Source Toolkit for Statistical Machine Translation.” Paper presented atAnnual Meeting of the Association for Computational Linguistics (ACL), demonstration session.
    [Google Scholar]
  19. Korea Institute of Patent Information (KIPI)
    Korea Institute of Patent Information (KIPI) (2018) “KIPRIS Brochure.” AccessedFebruary 17, 2022. eng.kipris.or.kr/enghome/kipris/kipris.jsp
  20. Korea Intellectual Property Office (KIPO) and Korea Institute of Patent Information (KIPI)
    Korea Intellectual Property Office (KIPO) and Korea Institute of Patent Information (KIPI) 2021 “Intellectual Property Statistics Service.” AccessedFebruary 17, 2022. ipstat.kipi.or.kr/cmm/main/mainPage.do
  21. Lavie, Alon, and Abhaya Agarwal
    2007 “METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments.” InProceedings of the Second Workshop on Statistical Machine Translation, Prague, Czech Republic, 23 June, ed. byChris Callison-Burch, Philipp Koehn, Christof Monz, and Cameron Shaw Fordyce, 228–231. Pennsylvania: Association for Computational Linguistics. 10.3115/1626355.1626389
    https://doi.org/10.3115/1626355.1626389 [Google Scholar]
  22. Lavie, Alon
    2013 “MT Evaluation: Human Measures and Assessment Methods.” AccessedFebruary 24, 2022. demo.clab.cs.cmu.edu/sp2013-11731
  23. Lee, Jieun, and Hyoeun Choi
    2023 “A Case Study of the Evaluation of Legal and Patent Korean-English Machine Translations by a Domain-Specific NMT Engine.” Journal of Translation Studies24(1): 9–37.
    [Google Scholar]
  24. Lommel, Arle
    2018 “Metrics for Translation Quality Assessment: A Case for Standardising Error Typologies.” InTranslation Quality Assessment, ed. byJoss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 109–208. Cham: Springer. 10.1007/978‑3‑319‑91241‑7_6
    https://doi.org/10.1007/978-3-319-91241-7_6 [Google Scholar]
  25. Long, Zi
    2016 “Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation.” InProceedings of the 3rd Workshop on Asian Translation (WAT2016), Osaka, Japan, 11–16 December, 47–57. Pennsylvania: Association for Computational Linguistics.
    [Google Scholar]
  26. Matsutani, Yohei
    2019 “Utilization of Machine Translation in the Japan Patent Office toward Improvement of Accessibility to Patent Information.” AccessedFebruary 17, 2022. www.aamtjapio.com/pslt2019/invited-talks
  27. Nurminen, Mary
    2019 “Decision-making, risk, and gist machine translation in the work of patent professionals.” InProceedings of the 8th Workshop on Patent and Scientific Literature Translation, Dublin, Ireland, 20 August, ed. byTakehito Utsuro, Katsuhito Sudoh, and Takashi Tsunakawa, 32–42. European Association for Machine Translation.
    [Google Scholar]
  28. 2020 “Raw Machine Translation Use by Patent Professionals: A Case of Distributed Cognition.” Translation, Cognition & Behaviour3 (1): 100–121. 10.1075/tcb.00036.nur
    https://doi.org/10.1075/tcb.00036.nur [Google Scholar]
  29. Olohan, Maeve
    2015Scientific and Technical Translation. London: Routledge. 10.4324/9781315679600
    https://doi.org/10.4324/9781315679600 [Google Scholar]
  30. Panic, Milica
    2020 “Automated MT Evaluation Metrics.” AccessedFebruary 17, 2022. https://blog.taus.net/automated-mt-evaluation-metrics
  31. Poibeau, Thierry
    2017Machine Translation. Cambridge: The MIT Press. 10.7551/mitpress/11043.001.0001
    https://doi.org/10.7551/mitpress/11043.001.0001 [Google Scholar]
  32. Popović, Maja
    2017 “Comparing Language Related Issues for NMT and PBMT between German and English.” Prague Bull Math Linguist108 (1): 209–220. 10.1515/pralin‑2017‑0021
    https://doi.org/10.1515/pralin-2017-0021 [Google Scholar]
  33. 2018 “Error Classification and Analysis for Machine Translation Quality Assessment.” InTranslation Quality Assessment, ed. byJoss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 129–158. Cham: Springer. 10.1007/978‑3‑319‑91241‑7_7
    https://doi.org/10.1007/978-3-319-91241-7_7 [Google Scholar]
  34. Premoli, Valeria, Elena Murgulo, and Diego Cresceri
    2019 “MTPE in Patents: A Successful Business Story.” InProceedings of MT Summit XVII, vol. 2, Dublin, Ireland, 19–23 August, edited byMikel Forcada, Andy Way, Barry Haddow, and Rico Sennrich, 36–41. European Association for Machine Translation.
    [Google Scholar]
  35. Qun, Liu, and Zhang Xiaojun
    2014 “Machine Translation.” InThe Routledge Encyclopedia of Translation Technology, ed. byChan Sin-Wai, 105–119. London: Routledge.
    [Google Scholar]
  36. Rossi, Laura, and Dion Wiggins
    2013 “Applicability and Application of Machine Translation Quality Metrics in the Patent Field.” World Patent Information351: 115–125. 10.1016/j.wpi.2012.12.001
    https://doi.org/10.1016/j.wpi.2012.12.001 [Google Scholar]
  37. Snover, Matthew
    2006 “A Study of Translation Edit Rate with Targeted Human Annotation.” InProceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, Cambridge, USA., 8–12 August. ed. byLaurie Gerber, 223–231. Association for Machine Translation in the Americas.
    [Google Scholar]
  38. Specia, Lucia, Dhwaj Raj, and Marco Turchi
    2010 “Machine Translation Evaluation versus Quality Estimation.” Machine Translation241: 39–50. 10.1007/s10590‑010‑9077‑2
    https://doi.org/10.1007/s10590-010-9077-2 [Google Scholar]
  39. Tinsley, John
    2012 “IP Translator: Facilitating Patent Search with Machine Translation.” InProceedings of the 10th Conference of the Association for Machine Translation in the Americas: Commercial MT User Program, San Diego, USA., 28 October–1 November. Association for Machine Translation in the Americas. AccessedJuly 31, 2022. https://aclanthology.org/2012.amta-commercial.17
    [Google Scholar]
  40. Toral, Antonio, and Víctor M. Sánchez-Cartagena
    2017 “A Multifaceted Evaluation of Neural versus Phrase-based Machine Translation for 9 Language Directions.” InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, 3–7 April, edited byMirella Lapata, Phil Blunsom, and Alexander Koller, 1063–1073. Pennsylvania: Association for Computational Linguistics. 10.18653/v1/E17‑1100
    https://doi.org/10.18653/v1/E17-1100 [Google Scholar]
  41. Tsai, Yvonne
    2017 “Linguistic Evaluation of Translation Errors in Chinese–English Machine Translations of Patent Titles.” Forum15 (1): 142–156. 10.1075/forum.15.1.08tsa
    https://doi.org/10.1075/forum.15.1.08tsa [Google Scholar]
  42. Wang, Xiaolin, Andrew, Finch Masao Utiyama, Taro Watanabe, and Eiichiro Sumita
    2014 “The NICT translation system for IWSLT 2014.” InProceedings of the 11th International Workshop on Spoken Language Translation: Evaluation Campaign, 139–142.
    [Google Scholar]
  43. Wang, Dan
    2009 “Chinese to English Automatic Patent Machine Translation at SIPO.” World Patent Information31(2): 137–139. 10.1016/j.wpi.2008.10.003
    https://doi.org/10.1016/j.wpi.2008.10.003 [Google Scholar]
  44. World Intellectual Property Organization (WIPO)
    World Intellectual Property Organization (WIPO) 2014 “Patent Cooperation Treaty: Yearly Review.” AccessedFebruary 17, 2022. https://www.wipo.int/publications/en/details.jsp?id=3253&plang=EN
  45. World Intellectual Property Organization
    World Intellectual Property Organization 2020a “WIPO Translate: Terms and Conditions for the Usage and User Guide.” AccessedFebruary 17, 2022. https://patentscope.wipo.int/translate/wtapta-user-manual-en.pdf
  46. World Intellectual Property Organization
    World Intellectual Property Organization 2020b “WIPO IP Facts and Figures 2020.” AccessedFebruary 17, 2022. https://www.wipo.int/publications/en/details.jsp?id=4533&plang=EN
  47. World Intellectual Property Organization
    World Intellectual Property Organization 2020c “World Intellectual Property Indicators 2020.” AccessedFebruary 17, 2022. https://www.wipo.int/publications/en/details.jsp?id=4526
  48. Ying, Cheng
    2021 “Errors of Machine Translation of Terminology in the Patent Text from English into Chinese.” ASP Transactions on Computers1 (1): 12–17. 10.52810/TC.2021.100022
    https://doi.org/10.52810/TC.2021.100022 [Google Scholar]
/content/journals/10.1075/forum.00030.lee
Loading
/content/journals/10.1075/forum.00030.lee
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was successful
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error