2026· Annual Meeting of the Association for Computational Linguistics· pp. 39175-39198· 0 citations· 46 references
Computer Science
TL;DR
This work proposes to synthesize alignment sequence pairs and fine-tune an encoder model with span alignment objective and introduces EXP - the first benchmark for explicit evaluation of label projection, thereby reducing confounders and non-determinism in method assessment.
Abstract
Information extraction (IE) systems rely on structured data for training, but such annotated data is highly imbalanced across languages, with low-resource languages receiving little attention. Label projection techniques aim to bridge this gap by transferring structured annotations from high-resource to low-resource languages. However, existing methods are either inaccurate or too slow for large-scale use. This work aims to address this problem by developing a more effective method that remains sufficiently efficient for large-scale projection. In particular, we propose to synthesize alignment sequence pairs and fine-tune an encoder model with span alignment objective, while controlling data influence during training. Experimental results across 50+ languages show that our framework consistently outperforms previous state-of-the-art methods while maintaining fast inference speed. In addition, we introduce EXP - the first benchmark for explicit evaluation of label projection, thereby reducing confounders and non-determinism in method assessment.
Sequence labeling is a fine-grained information extraction task, yet existing large language model-based approaches suffer from insufficient domain alignment and low inference efficiency. To address these issues, we propose DIRECT, a framework that addresses these issues through training-time optimization and inference-time rectification. Specifically, DIRECT performs Direct Preference Optimization (DPO) after supervised fine-tuning to strengthen task alignment with human preferences, and introduces a controlled decoding process that enforces fixed output formats and restricts predictions to candidate sets. To further improve efficiency, a template-filling mechanism requires the model to generate only label tokens while reusing prefixed content through the KV Cache, thus reducing redundant computation. Experimental results on eight datasets demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared to existing methods.
Yilei Wang, Jiaxin Gan, Kexuan Zhang et al.· 0 citations
This paper proposes an alignment-aware multi-granularity tagging framework. First, this method uses a cross-lingual pre-trained model to encode source and target language contexts jointly while explicitly modeling-level bilingual correspondences via a learnable soft alignment layer. Second, a gated local enhancement module is introduced to dynamically fuse n-gram-level surface features on top of the Transformer high-level representation, preserving fine-grained error signals that are easily smoothed by attention mechanisms. Finally, a label-aware focus loss function is designed to alleviate the extreme imbalance between positive and negative samples. This function supports joint prediction of four types of fine-grained error labels. Results showed that on the large-scale benchmark of WMT QE Shared Task 2023, covering six language pairs, alignment-aware multi-granularity tagging achieved 78.6% in Micro-F1, 72.3% in Macro-F1, and improved Recall@Rare to 62.3%, especially excelling in low-frequency error types and significantly outperforming the baseline model.
Bijuan Wang, Lingli Zhu, Hongli Wen· International Journal of Inf...· 0 citations
Large Language Models (LLMs) have achieved remarkable performance largely due to the availability of high-quality and large-scale training data. However, this reliance creates a significant performance disparity between high-resource languages such as English and low-resource languages like Arabic. The imbalance in data exposure during the pre-training phase persists into the bilingual supervised fine-tuning (SFT) stage, limiting the models’ semantic depth and reasoning capabilities in low-resource languages.To address the aforementioned imbalance, we depart from traditional approaches by computing the optimal learning rates for each language in the dataset separately and transforming these learning rates into language-specific weighting coefficients used in the loss function. Experimental results obtained on the XQuAD test set after training on the MURI dataset demonstrate that the proposed method preserves performance in English while significantly improving Arabic performance, achieving 56.36 F1 and 40.27 Exact Match (EM) scores.
D. Özcan, Çağrı Yeşil, Hasan Saribas et al.· Signal Processing and Commun...· 0 citations
Centroid Intervention Fusion is proposed, a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator and outperforms the strongest prior pairwise intervention baseline by up to +3.3% across four model backbones.
PuRF is introduced, a novel PuRiFication-driven cache-based method for multi-label test-time adaptation of vision-language models that consistently outperforms state-of-the-art methods on ViT-B/32 across five datasets.
Yiwen Liang, Hui Chen, Yizhe Xiong et al.· 0 citations
Neural machine translation (NMT) has witnessed substantial advancements, leveraging its learning capability to deliver highly accurate translations. Nevertheless, the efficacy of NMT models is contingent upon the accessibility of extensive-scale, high-quality training data, and its performance suffers notably in the absence of such datasets. To tackle this challenge, we propose a semantic distance augmentation (SDA) method that integrates syntactic information from constituency parse trees into the NMT encoder to optimize self-attention. Specifically, the source language sentences in the training set are analyzed by constituency parse analysis and the semantic distance attention matrix is constructed. Then, a fusion strategy is designed to integrate this matrix into the self-attention weight, enhancing the representation of the source sentences. In addition, a SDA length-aware strategy is proposed to adaptively control the contribution of semantic distance in the attention computation. Empirical evaluations across multiple low-resource language pairs reveal that the SDA method achieves statistically significant improvements in translation quality over the strong baseline, without requiring additional training data or increasing model complexity.
Fuxue Li, Hong Yan, Chuncheng Chi et al.· PeerJ Computer Science· 0 citations