Beyond Rankings: Insights from ALTAE’s PTT-MULTITAN Submission to the WMT 2026 Shared Task on Automated Translation Quality Evaluation Systems
Abstract
We describe PTT - MULTITAN , our submission to the WMT26 MT Evaluation shared task, Task 3 (error-free segment detection). The task requires a binary decision – whether a machine translation (MT) contains any error – without access to reference translations, and is scored using the Matthews Correlation Coefficient (MCC). Our system combines a supervised classifier trained on WMT25 human error-span annotations with three families of reference-free indicators: structural and phraseological cues, corpus-linguistic association measures computed over an LLM-as-judge model, and format/locale conformity checks. Our improved submission raises MCC from 0 . 159 to 0 . 168 . We submitted six language pairs from the official WMT26 test set: English into Belarusian, German, Russian, Ukrainian, Simplified Chinese, and Traditional Chinese. Overall, TRIVE emerges as the strongest general-purpose system, with the highest full-coverage score and strong performance across European languages. Correspondence Analysis (CA) offers complementary insights into lexical and domain differences, while future evaluation should involve translators and translation scholars.