Cross-Lingual Triplet Data Engineering for Reference-Free Quality Estimation Domain Adaptation in Historical Corpus

Abstract

Automatic reference-free Quality Estimation (QE) for machine translation in domain-specific contexts, such as historical corpora, requires adaptation to text characteristics that are rarely represented in general-purpose metric training data. This challenge is particularly pronounced for low-tomid resource language pairs such as Dutch–Indonesian, where human-annotated QE data are scarce. To address the scarcity of human-annotated QE data, this study empirically evaluates an English-pivoted cross-lingual triplet strategy for adapting trainable Neural QE models within the COMET framework, using semantically related Dutch–Indonesian, Dutch–English, and English-Indonesian sentence pairs as additional learning signals beyond the direct Dutch–Indonesian baseline. We hypothesize that incorporating these triplet inputs improves the predictive performance of trainable QE models over the direct baseline. The hypothesis is evaluated within an InfoXLM-based COMET framework supervised by COMETKiwi22-DA pseudolabels, with two trainable architectures: a Referenceless model with late fusion and a Unify model with early fusion of the source and its translation. Models are trained using 5-fold crossvalidation on 5,723 sentences from the book Talen van geweld, with frozen embeddings and low-rate encoder fine-tuning, and evaluated on two 1,000-sentences hold-out test sets generated by Google Translate and the SEA-LION LLM. Experimental results show that the triplet scheme produces a statistically significant improvement in Spearman rank correlation in the Unify Model, with an improvement of up to +0.079 over the direct baseline. The performance gain is substantially less pronounced in the Referenceless Model. The Unify model also exhibits stable performance across the two translation engines, with a smaller reduction in correlation under the triplet condition than under the direct baseline. These findings provide empirical evidence that semantically related triplet inputs can improve the predictive performance of trainable COMET-based QE models, with the magnitude of the improvement varying across model architectures.