liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
On the practice of error analysis for machine translation evaluation
Linköping University, Department of Computer and Information Science, NLPLAB - Natural Language Processing Laboratory. Linköping University, The Institute of Technology.
Linköping University, Department of Computer and Information Science, NLPLAB - Natural Language Processing Laboratory. Linköping University, The Institute of Technology.
2012 (English)In: Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), European Language Resources Association , 2012, 1786-1790 p.Conference paper, Published paper (Refereed)
Abstract [en]

Error analysis is a means to assess machine translation output in qualitative terms, which can be used as a basis for the generation of error profiles for different systems. As for other subjective approaches to evaluation it runs the risk of low inter-annotator agreement, but very often in papers applying error analysis to MT, this aspect is not even discussed. In this paper, we report results from a comparative evaluation of two systems where agreement initially was low, and discuss the different ways we used to improve it. We compared the effects of using more or less fine-grained taxonomies, and the possibility to restrict analysis to short sentences only. We report results on inter-annotator agreement before and after measures were taken, on error categories that are most likely to be confused, and on the possibility to establish error profiles also in the absence of a high inter-annotator agreement.

Place, publisher, year, edition, pages
European Language Resources Association , 2012. 1786-1790 p.
Keyword [en]
Mahine translation, statistical machine translation, error analysis, inter-annotator agreement
National Category
Language Technology (Computational Linguistics) General Language Studies and Linguistics Computer Science
Identifiers
URN: urn:nbn:se:liu:diva-80353ISI: 000323927701143ISBN: 978-2-9517408-7-7 (print)OAI: oai:DiVA.org:liu-80353DiVA: diva2:546565
Conference
The Eight International Conference on Language Resources and Evaluation (LREC'12)
Available from: 2012-08-23 Created: 2012-08-23 Last updated: 2015-05-28

Open Access in DiVA

No full text

Other links

http://www.lrec-conf.org/proceedings/lrec2012/pdf/717_Paper.pdf

Authority records BETA

Stymne, SaraAhrenberg, Lars

Search in DiVA

By author/editor
Stymne, SaraAhrenberg, Lars
By organisation
NLPLAB - Natural Language Processing LaboratoryThe Institute of Technology
Language Technology (Computational Linguistics)General Language Studies and LinguisticsComputer Science

Search outside of DiVA

GoogleGoogle Scholar

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 63 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf