liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
ELEXR: Automatic Evaluation of MachineTranslation Using Lexical Relationships
University of Tehran, Iran.
University of Tehran, Iran.
University of Tehran, Iran.
Linköping University, Department of Computer and Information Science, Human-Centered systems. Linköping University, The Institute of Technology. (CILTLAB)ORCID iD: 0000-0003-1942-6063
2013 (English)In: Advances in Artificial Intelligence and Its Applications: 12th Mexican International Conference on Artificial Intelligence, MICAI 2013, Mexico City, Mexico, November 24-30, 2013, Proceedings, Part I / [ed] Félix Castro, Alexander Gelbukh, Miguel González, Springer Berlin/Heidelberg, 2013, 394-405 p.Conference paper, Published paper (Refereed)
Abstract [en]

This paper proposes ELEXR, a novel metric to evaluate machine translation (MT). In our proposed method, we extract lexical co-occurrence relationships of a given reference translation (Ref) and its corresponding hypothesis sentence using hyperspace analogue to language space matrix. Then, for each term appearing in these two sentences, we convert the co-occurrence information into a conditional probability distribution. Finally, by comparing the conditional probability distributions of the words held in common by Ref and the candidate sentence (Cand) using Kullback-Leibler divergence, we can score the hypothesis. ELEXR can evaluate MT by using only one Ref assigned to each Cand without incorporating any semantic annotated resources like WordNet. Our experiments on eight language pairs of WMT 2011 submissions show that ELEXR outperforms baselines, TER and BLEU, on average at system-level correlation with human judgments. It achieves average Spearman’s rho correlation of about 0.78, Kendall’s tau correlation of about 0.66 and Pearson’s correlation of about 0.84, corresponding to improvements of about 0.04, 0.07 and 0.06 respectively over BLEU, the best baseline.

Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2013. 394-405 p.
Series
Lecture Notes in Computer Science, ISSN 0302-9743 (print), 1611-3349 (online) ; 8265
Keyword [en]
Machine Translation, Natural Language Processing, Automatic Evaluation, Evaluation Metrics, BLEU, TER, Lexical Relationships, Machine Translation Evaluation
National Category
Computer Science
Identifiers
URN: urn:nbn:se:liu:diva-107633DOI: 10.1007/978-3-642-45114-0_32ISBN: 978-3-642-45113-3 (print)ISBN: 978-3-642-45114-0 (print)OAI: oai:DiVA.org:liu-107633DiVA: diva2:726135
Conference
12th Mexican International Conference on Artificial Intelligence, MICAI 2013, Mexico City, Mexico, November 24-30, 2013
Available from: 2014-06-17 Created: 2014-06-17 Last updated: 2014-11-28Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textFind book at a Swedish library/Hitta boken i ett svenskt bibliotekFind book in another country/Hitta boken i ett annat land

Authority records BETA

Maleki, Jalal

Search in DiVA

By author/editor
Maleki, Jalal
By organisation
Human-Centered systemsThe Institute of Technology
Computer Science

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 26 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf