Skip to content

FACET at WMT 2026 Automated Translation Quality Evaluation Task

Sep 2026 · 0 citations · 31 references
Computer Science

TL;DR

FACET, the authors' reference-free submission to the WMT26 Automated Translation Quality Evaluation Task, is presented, which decomposes evaluation into Fluency, Accuracy, and Consistency passes and gives each pass only the context its error type requires.

Abstract

Different error types in machine translation require different evidence. Whether meaning is preserved can be judged only against the source, while whether the target is well-formed, or whether it names one entity consistently, can be judged from the target alone. We present FACET, our reference-free submission to the WMT26 Automated Translation Quality Evaluation Task, which decomposes evaluation into Fluency, Accuracy, and Consistency passes and gives each pass only the context its error type requires. A single fixed model is prompted three times, and the merged error spans yield the three task outputs, error spans, quality scores, and error-free labels, with no trained components. We also submit FACET-C, which omits the Consistency pass. Without gold labels, we characterize the predictions of FACET. Its system rankings place post-edited human translation first, and the Consistency pass changes about a tenth of segment scores while leaving the ranking nearly unchanged.

View source

Similar papers

#natural language process... Preprint Sep 2026

In the Blind: Building Pseudo-References for MT Evaluation

The WMT26 General MT task evaluates systems on 10 language pairs that have no human references (neither translated from scratch nor post-edited from MT output by humans) and how the pseudo-references were built is described.

Diptesh Kanojia, Chi-Kiu Lo, Archchana Sindhujan et al. · 0 citations
#natural language process... Preprint Sep 2026

SalamandraTA at WMT 2026 Terminology Shared Task: Hard Examples Are Better Teachers

Terminology-aware translation asks for more than a correct translation: the output must use the exact terms a glossary prescribes. The standard recipe, fine-tuning on glossary-annotated translation pairs, hides an inefficiency: for most examples the glossary prescribes exactly what the model would have produced anyway,...

Xi-Xian Liao, Maite Melero · 0 citations
#natural language process... Preprint Sep 2026

Discourse Dependency: A Continuous Criterion for Translation Difficulty

This work argues that one meaningful and currently unmeasured axis is referential reach, the distance a segment must look back into its document to resolve the entities and pronouns it contains, and formalizes this as discourse dependency (DDP), a metric-free, source-side measure computed from named entity re-mentions...

Ahrii Kim, Chanjun Park, Seong-heum Kim · 0 citations

Rethinking Human-Aligned Evaluation: An Analysis of Semantic Metrics Beyond WER

The results support shifting ASR evaluation toward CER both for English and for morphosyllabic writing systems as it is a more interpretable and low-cost metric for what evaluation should actually capture, and using SemDist as a complementary evaluation.

Hritika Sharma, Thibault Bañeras-Roux, Alessandra Pinto et al. · 0 citations
#natural language process... Preprint Sep 2026

Improving Term Evaluation in Machine Translation: Variation Matters

It is argued for variation-aware evaluation that conditions consistency penalties on whether target-side variation mirrors source-side variation, and constraining MT with a glossary improves accuracy and consistency but degrades CTV by suppressing valid variation.

Nicolas M. Dahan, Ziqian Peng, François Yvon et al. · 1 citation · ⚡1

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.