In Machine Translation (MT) evaluation plays a key role both in their development and improvement of the systems. BLEU is one of the most well-known and widely used but BLEU has its weaknesses regarding translation quality and its tendency to favour statistically-based MT systems. VERTa is trying to address the evaluation of the MT from a more linguistically-motivated point of view.
VERTa is part of the research that intends to emphasize the effectiveness of linguistic analysis in order to identify and test those linguistic features that help in evaluating traditional concepts of adequacy and fluency.
VERTA combines different modules: Lexical module, Morphological, Syntactic module, Ngram module, Semantic module and can be easily adapted to different evaluation types (fluency, adequacy, MT quality) and to different languages or genres.
We have focused on MT output in English (2014 WMT competition as part of the ACL 2014 WMT workshop) with an application to Spanish so as to test the portability of our approach.