Skip to content
Conference Open access

Beyond Rankings: Insights from ALTAE’s PTT-MULTITAN Submission to the WMT 2026 Shared Task on Automated Translation Quality Evaluation Systems

2026 · Proceedings of the Eleventh Conference on Machine Translation · 0 citations · 17 references

Abstract

We describe PTT - MULTITAN , our submission to the WMT26 MT Evaluation shared task, Task 3 (error-free segment detection). The task requires a binary decision – whether a machine translation (MT) contains any error – without access to reference translations, and is scored using the Matthews Correlation Coefficient (MCC). Our system combines a supervised classifier trained on WMT25 human error-span annotations with three families of reference-free indicators: structural and phraseological cues, corpus-linguistic association measures computed over an LLM-as-judge model, and format/locale conformity checks. Our improved submission raises MCC from 0 . 159 to 0 . 168 . We submitted six language pairs from the official WMT26 test set: English into Belarusian, German, Russian, Ukrainian, Simplified Chinese, and Traditional Chinese. Overall, TRIVE emerges as the strongest general-purpose system, with the highest full-coverage score and strong performance across European languages. Correspondence Analysis (CA) offers complementary insights into lexical and domain differences, while future evaluation should involve translators and translation scholars.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.