CTFAlign is introduced, a lightweight, training-free approach for document-level word alignment that applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions and introduces MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior.
Abstract
Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.
Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single granularity, so practitioners often need separate systems for word- and sentence-level alignment---especially in multilingual and long-text settings. We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment with a single lightweight model. Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices, and obtains document-level $m$--$n$ sentence alignments via sentence embeddings combined with dynamic programming. To balance fine-grained alignment accuracy and sentence-representation quality, we use a four-stage training pipeline: alignment-oriented continued pre-training, self-supervised learning, supervised fine-tuning on human annotations, and sentence-embedding distillation from a strong multilingual teacher. Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks and generalizes well to unseen language pairs. Surprisingly, later-stage supervised fine-tuning on short texts further improves alignment quality while retaining the long-context understanding acquired in earlier training, keeping the model robust on long-text word alignment. \normalsize {\color{blue}\textbf{Code}: https://github.com/MilkDargon/OmniAlign}\par {\color{blue}\textbf{Model}: https://huggingface.co/WPS-Qingqiu/OmniAlign}
Mengpeng Yang, Jingxu Yang, Chao Chen et al.· 0 citations
Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently suffers from structural misalignment, manifesting as sentence omissions or hallucinations that violate the core requirement of source-target correspondence. To address this, we introduce Sentence Translation Alignment Rate (STAR), an auxiliary metric that explicitly quantifies sentence-level structural fidelity. Building on this, we propose STAR-masked Preference Optimization (StarPO), a framework that ranks document-level hypotheses by structural quality and utilizes a dynamic alignment mask to focus optimization on misaligned segments. Experimental results across news and literary domains demonstrate that StarPO significantly enhances translation quality and structural integrity. Notably, StarPO allows compact models to surpass the performance of massive proprietary systems like GPT-4o while maintaining superior token efficiency.
Yichen Dong, Hao Wang, Junhui Li et al.· 0 citations
It is confirmed that cross-lingual alignment can emerge from the structure of language and the information it carries rather than from joint training, and this points to practical future directions including model stitching, merging, and modular multilingual systems built from monolingual components.
Ej Zhou, Suchir Salhan, Catherine Arnett et al.· 0 citations
In highly specialized and terminology-dense scientific and technical texts, existing word sense disambiguation (WSD) models struggle to adequately model the contextual semantic dependencies of polysemous words, especially in engineering domains where the same term may carry different technical meanings across contexts. To address this issue, this paper proposes a robust WSD model that integrates a bidirectional long shortterm memory network (BiLSTM ) with an attention mechanism, specifically designed for Chinese patent texts. First, a two-layer BiLSTM is used for bidirectional context modeling to capture long-range dependencies. Then, a multi-head attention mechanism uses dynamic weighting to highlight key semantic components, generating highly discriminative context vectors. Finally, a paraphrase alignment mechanism employs bilinear matching to align the context vectors with candidate paraphrase embeddings, thereby reducing semantic confusion. Experiments show that the model achieves a Top-1 accuracy of 88.6 % on high-frequency words, with an average paraphrase alignment similarity of 0.920. In perturbation tests, the average robustness index is 0.159, representing reductions of 62.7%, 53.9%, and 39.5% compared to Word2Vec+CNN, BiLSTM, and BERT, respectively. The method presented in this paper helps to enhance the accuracy and stability of word meaning recognition in technical texts, providing reliable support for knowledge mining and intelligent text processing in technical domains. Its terminology alignment is also useful for engineering corpora where antenna, wavepropagation and materials terms require context-sensitive interpretation.
L. X. Gao, T. Dong, M. H. Yang· Advanced Electromagnetics· 0 citations
A PMI-based translation metric is proposed, which is less dependent on the target language and correlates strongly with chrF, and finds that CLA with English predicts translation quality comparably to or better than source-target CLA.
Adnan Al Ali, Kathy Hämmerl, Jindrich Libovický et al.· 0 citations
This work releases LAMAR, a language aware multilingual cross encoder trained to account for both semantic relevance and language coherence, which achieves the best performance overall and across all languages examined individually on general multilingual reranking benchmarks.
Seongtae Hong, Youngjoon Jang, Jungseob Lee et al.· 0 citations