The results show that the quality of small ``base'' models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning.
Abstract
Neural machine translation (NMT) in the legal domain is a linguistically and conceptually demanding task, primarily due to the complexity of legal language and the high level of precision it requires. The recent emergence of reasoning-capable language models opens new possibilities for tackling such challenges. They add to a set of other previously proposed techniques to enhance the translation quality, which includes supervised fine-tuning and reinforcement learning. In this work, we perform a comparison between these various approaches. More particularly, we evaluate small language models such as Qwen3.5 4B, Qwen3.5 9B, and Gemma 3 12B enhanced with various re-training paradigms and compare their performances against frontier reasoning models. We focus on the Swiss legal system, which -- with its unique multilingual statutes -- offers a particularly challenging testbed for reasoning-augmented models. Our results show that the quality of small ``base''models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning. The performance of enhanced small models is close to the one of state-of-the-art reasoning models yet remains inferior. We also note that re-training paradigms yield diminishing returns as model size increase. The code and models are publicly available at https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL.
This work investigates the importance of including the model's reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases and shows that including the reasoning, specifically during inference, has a positive effect on the overall translation quality.
Michael Jungo, Aixiu An· Proceedings of the 2026 ACM...· 0 citations
A novel fragment-based reasoning framework is introduced in which the model first extracts parallel source-target fragments from retrieved similar exemplars, and uses these fragments as intermediate reasoning traces to produce the final translation.
Maxime Bouthors, J. Crego, François Yvon· 0 citations
This study evaluates the applicability of generative artificial intelligence (AI), specifically Large Language Models (LLMs), in legal assistance tasks using the Artificial Intelligence for Legal Assistance challenge dataset, which includes 197 statutes, 2914 judicial cases, and 50 legal queries. The research compared conversational LLM tools with semantic retrieval systems based on dense embeddings. Results show that general-purpose tools (ChatGPT, NotebookLM) achieved poor discrimination, while embedding-based methods significantly improved accuracy: the OpenAI text-embedding-ada-002 model reached 46.43% retrieval accuracy, and a fine-tuned all-mpnet-base-v2 model improved from 18.9% to 31.87% (a 68.6% relative gain). These findings demonstrate that combining LLMs with semantic retrieval techniques enhances precision and scalability in legal information systems, offering a viable roadmap for developing domain-specific, efficient, and sustainable AI legal assistants.
Carlos A. Martín, J. Torres, R. M. Aguilar et al.· Logic Journal of the IGPL· 0 citations
Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resource languages, with English and Chinese dominating both pre-training corpora and evaluation suites. The recently released PolyMath (Wang et al., 2025) dataset represents a significant step forward, yet its coverage is still limited to 18 only high-resource languages. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional {underrepresented languages spanning 6 language families -- ranging from mid-resource to extreme low-resource settings. We constructed the dataset through a human-curated pipeline, where native speakers thoroughly validated pre-computed translations. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales -- small, mid-size, large, and closed-source ensembles -- probing the multilingual mathematical reasoning capabilities of state-of-the-art models under diverse linguistic conditions. Our fine-grained analysis confirms a persistent gap in mathematical reasoning performance between high-resource and underrepresented languages, with stronger results largely associated with better instruction-following ability. We fully open-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities.
Daryna Dementieva, N. Babakov, Kathy Hammerl et al.· 0 citations
Machine translation (MT) technologies are currently undergoing a paradigm shift, transitioning
from specialized Neural Machine Translation (NMT) frameworks to the broader capabilities of
Large Language Models (LLMs). This paper examines the current standing of the Croatian language
within this technological evolution.
While bilingual NMT models often exhibit high precision, multilingual NMT leverage transfer
learning to enhance performance for low–resource language pairs, but with lower performance for
high–resource ones. Conversely, LLMs—whether general–purpose or fine–tuned for translation—
offer superior multilingual proficiency and context awareness. Unlike NMT, LLMs can process extended discourse, such as full paragraphs or documents, leading to significant improvements in
coreference resolution and gender agreement. Despite the substantial computational requirements
of LLMs, recent optimization techniques allow for smaller, more efficient versions that maintain
high output quality.
This study evaluates the performance of various NMT and LLM architectures specifically for
Croatian from/to English and Spanish using several automatic quality evaluation metrics. The findings demonstrate that open–source models can achieve, and occasionally surpass, the quality of
Google Translate, a widely used commercial NMT system. Furthermore, while our evaluation focuses on this specific language triad, the multilingual nature of the analysed systems suggests that
open–source models provide high–quality translation capabilities for Croatian across dozens, if not
hundreds, of language pairs.
Antoni Oliver, Sergi Álvarez–Vidal· Suvremena Lingvistika· 1 citation
PAMT is proposed, a process-aligned training framework that combines cold-start domain-aware Long-CoT supervision with reinforcement learning that improves over base models, outperforms MT-specialized baselines on average, and remains competitive with strong LLMs/LRMs across in-domain, OOD, and multilingual settings.
Yongshi Ye, Biao Fu, Chongxuan Huang et al.· 0 citations
