This paper proposes a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models, and reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance.
Abstract
The adoption of large pre-trained multilingual models for neural machine translation (MNMT) faces a major challenge: excessive memory and computational consumption due to overly large vocabularies and embedding layers. Although existing compression methods like pruning, quantization and knowledge distillation reduce parameter redundancy, they mainly preserve the structure of the original vocabulary, thereby leaving a major source of inefficiency unresolved. We propose in this paper a general optimization framework that combines a vocabulary pruning method with a targeted fine-tuning protocol for MNMT models. We evaluate the proposed framework using three models (M2M100, NLLB-200, mBART-50) on the English-Arabic language pair. Our approach reduces the vocabulary size from over 128,000 to approximately 10,000 tokens, enabling a 60% memory saving without any loss in performance. Results show that optimized multilingual models can match or exceed the performance of dedicated bilingual baselines. In particular, the pruned and fine-tuned M2M100 model achieves a competitive BLEU score of 42.04 (against 44.59 for the OPUS-MTen- ar bilingual model) while it significantly outperforms it on the COMET metric (0.8730 vs 0.7911) revealing superior semantic adequacy and fluency.
The results show that multilingual transfer is the dominant factor in extremely low-resource Bantu translation while eliminating the need for heuristic proxy selection, and all systems fail to preserve tonal diacritics, highlighting an open challenge.
Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu et al.· 0 citations
The effectiveness of multilingual transfer learning in low-resource settings is demonstrated by the fine-tuned Multilingual Bidirectional and Auto-Regressive Transformer-50 model, significantly outperforming the pretrained baseline.
G. Harshitha, Vasudeva, Nisha P. Poojary et al.· Engineering, Technology &...· 0 citations
The results indicate that a moderately sized, shared self-attention architecture can deliver production-quality multilin-gual translation within the resource constraints of an academic de-ployment, while surfacing clear directions – low-resource language coverage, domain adaptation, and speech-based extension – for con-tinued development.
Darshan Gowda D H and Dr. Kruti R· International Journal of Adv...· 0 citations
Large Language Models (LLMs) have achieved remarkable performance largely due to the availability of high-quality and large-scale training data. However, this reliance creates a significant performance disparity between high-resource languages such as English and low-resource languages like Arabic. The imbalance in data exposure during the pre-training phase persists into the bilingual supervised fine-tuning (SFT) stage, limiting the models’ semantic depth and reasoning capabilities in low-resource languages.To address the aforementioned imbalance, we depart from traditional approaches by computing the optimal learning rates for each language in the dataset separately and transforming these learning rates into language-specific weighting coefficients used in the loss function. Experimental results obtained on the XQuAD test set after training on the MURI dataset demonstrate that the proposed method preserves performance in English while significantly improving Arabic performance, achieving 56.36 F1 and 40.27 Exact Match (EM) scores.
D. Özcan, Çağrı Yeşil, Hasan Saribas et al.· Signal Processing and Commun...· 0 citations
Recent progress in artificial intelligence has pushed machine translation to high levels of accuracy for widely resourced languages. Yet for many indigenous and endangered languages, comparable tools remain absent, largely because digitized linguistic data are scarce. Kibajuni, a minimally documented Bantu language spoken along the Kenyan coast, illustrates this gap. Publicly accessible English–Kibajuni machine translation systems are not available, which restricts both everyday digital use and broader language preservation work.This paper reports the design, construction, and assessment of a compact Transformer-based Neural Machine Translation (NMT) system for English–Kibajuni translation. Training relied on a community-produced parallel corpus of roughly 10,000 aligned sentence pairs. A Design Science Research approach guided development of the full translation workflow, beginning with corpus preparation and continuing through Byte Pair Encoding (BPE) tokenization, a custom encoder–decoder Transformer, supervised training in PyTorch, beam-search decoding at inference time, and deployment as a web application.Because data were limited, emphasis was placed on training stability and generalization rather than increasing model size. The system therefore integrated AdamW, OneCycle learning-rate scheduling, dropout, label smoothing, gradient clipping, mixed-precision training, and early stopping driven by validation BLEU. Results indicate that, despite the small dataset, the model learned usable semantic correspondences between English and Kibajuni while remaining computationally light. The final network contains about 6–8 million parameters, occupies roughly 27 MB, and supports real-time translation on modest hardware.
In practical terms, the work provides one of the earliest operational English–Kibajuni neural translation platforms. At the methodological level, it offers a reproducible template for developing MT systems for other under-resourced African languages. Taken together, the findings suggest that appropriately scaled Transformer models, paired with subword tokenization and carefully tuned training procedures, can materially advance digital inclusion and language preservation for endangered languages.
Anwar H. Ahmed, W. Bana, Mathew M. Egessa et al.· SOUTH SAHARA MULTIDISCIPLINA...· 0 citations