Open this publication in new window or tab >>2013 (English)In: Advances in Artificial Intelligence and Its Applications: 12th Mexican International Conference on Artificial Intelligence, MICAI 2013, Mexico City, Mexico, November 24-30, 2013, Proceedings, Part I / [ed] Félix Castro, Alexander Gelbukh, Miguel González, Springer Berlin/Heidelberg, 2013, p. 394-405Conference paper, Published paper (Refereed)
Abstract [en]
This paper proposes ELEXR, a novel metric to evaluate machine translation (MT). In our proposed method, we extract lexical co-occurrence relationships of a given reference translation (Ref) and its corresponding hypothesis sentence using hyperspace analogue to language space matrix. Then, for each term appearing in these two sentences, we convert the co-occurrence information into a conditional probability distribution. Finally, by comparing the conditional probability distributions of the words held in common by Ref and the candidate sentence (Cand) using Kullback-Leibler divergence, we can score the hypothesis. ELEXR can evaluate MT by using only one Ref assigned to each Cand without incorporating any semantic annotated resources like WordNet. Our experiments on eight language pairs of WMT 2011 submissions show that ELEXR outperforms baselines, TER and BLEU, on average at system-level correlation with human judgments. It achieves average Spearman’s rho correlation of about 0.78, Kendall’s tau correlation of about 0.66 and Pearson’s correlation of about 0.84, corresponding to improvements of about 0.04, 0.07 and 0.06 respectively over BLEU, the best baseline.
Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2013
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 8265
Keywords
Machine Translation, Natural Language Processing, Automatic Evaluation, Evaluation Metrics, BLEU, TER, Lexical Relationships, Machine Translation Evaluation
National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-107633 (URN)10.1007/978-3-642-45114-0_32 (DOI)978-3-642-45113-3 (ISBN)978-3-642-45114-0 (ISBN)
Conference
12th Mexican International Conference on Artificial Intelligence, MICAI 2013, Mexico City, Mexico, November 24-30, 2013
2014-06-172014-06-172018-01-23Bibliographically approved