Error profiling for evaluation of machine-translated text: a Polish-English case study
2012 (English)In: Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12) / [ed] Nicoletta Calzolari (Conference Chair) and Khalid Choukri and Thierry Declerck and Mehmet Uğur Doğan and Bente Maegaard and Joseph Mariani and Jan Odijk and Stelios Piperidis, Paris, Frankrike: European Language Resources Association (ELRA) , 2012, 1764-1770 p.Conference paper, Presentation (Other academic)
We present a study of Polish-English machine translation, where the impact of various types of errors on cohesion and comprehensibility of the translations were investigated. The following phenomena are in focus: (i) The most common errors produced by current state-of-the-art MT systems for Polish-English MT. (ii) The effect of different types of errors on text cohesion. (iii) The effect of different types of errors on readers' understanding of the translation. We found that errors of incorrect and missing translations are the most common for current systems, while the category of non-translated words had the most negative impact on comprehension. All three of these categories contributed to the breaking of cohesive chains. The correlation between number of errors found in a translation and number of wrong answers in the comprehension tests was low. Another result was that non-native speakers of English performed at least as good as native speakers on the comprehension tests.
Place, publisher, year, edition, pages
Paris, Frankrike: European Language Resources Association (ELRA) , 2012. 1764-1770 p.
Machine Translation, Evaluation methodologies, Profiling
IdentifiersURN: urn:nbn:se:liu:diva-80276ISBN: 978-2-9517408-7-7OAI: oai:DiVA.org:liu-80276DiVA: diva2:546399
Eight International Conference on Language Resources and Evaluation (LREC'12), 21-27 May 2012, Istanbul, Turkey