Results demonstrate that integrating a rule-based virtual legal assistant with contrastive retrieval and preference optimization provides an effective and computationally efficient strategy for aligning the legal reasoning capabilities of small language models, offering a scalable solution for legal question answering in specialized low-resource environments.
Abstract
Deploying pre-trained language models for question answering in specialized low-resource legal domains remains challenging due to the scarcity of expert-annotated data, the complexity of legal reasoning, and limited computational resources. Existing retrieval-augmented generation approaches improve factual grounding but often fail to preserve logically consistent legal reasoning when retrieved evidence is noisy, while reinforcement learning-based alignment methods incur substantial computational costs. This study proposes an automated end-to-end framework that integrates contrastive retrieval with context-aware preference alignment to enhance the legal reasoning performance of small language models. The framework employs an evolutionary instruction-tuning methodology to systematically mutate seed statutes, generating a diverse synthetic corpus conditioned on structural legal reasoning to address data scarcity. We introduce a multi-objective rule-based reward model that functions as a virtual legal assistant by jointly evaluating logical entailment and statutory faithfulness. The resulting reward scores are used to automatically construct preference pairs for Context Direct Preference Optimization, enabling efficient policy alignment while substantially reducing the need for costly human annotation. The proposed framework is evaluated on a comprehensive benchmark comprising expert-validated queries derived from the Vietnamese Civil Code. Experimental results demonstrate consistent improvements across all evaluated baselines, achieving a 13.5% relative improvement in citation error rates over the strongest standard DPO baseline, while also demonstrating a 66% reduction compared to zero-shot prompting. The proposed framework further achieves superior performance in logical faithfulness (0.79), answer relevance (0.85), semantic similarity (0.855), and expert human evaluation (4.5/5). These findings demonstrate that integrating a rule-based virtual legal assistant with contrastive retrieval and preference optimization provides an effective and computationally efficient strategy for aligning the legal reasoning capabilities of small language models, offering a scalable solution for legal question answering in specialized low-resource environments.
Retrieval-augmented generation (RAG) improves question answering by grounding large language models (LLMs) in external knowledge such as text corpora. However, its reasoning process remains largely opaque: intermediate reasoning steps are difficult to verify and cannot be reliably attributed to specific evidence. Moreover, missing user-specific context is rarely detected systematically, often leading to incomplete or incorrect output. We propose NeSy-RAG, a modular neuro-symbolic RAG framework that synthesizes attributable Prolog modules from retrieved text chunks. For each chunk, the system generates semantically meaningful predicates that encode Boolean claims, which may depend on user facts. Using joint natural language-code embeddings, predicates are retrieved and composed into Prolog queries. To address incomplete user context, we introduce a symbolic knowledge-gap detection mechanism that identifies missing user facts whose truth values affect the query outcome and automatically triggers follow-up interactions. Executing the resulting Prolog queries yields deterministic answers together with transparent execution traces that link each reasoning step to its originating source. On the ShARC benchmark, without domain-specific training, NeSy-RAG achieves 61.1% accuracy, outperforming a same-model RAG baseline that achieves 42.8% accuracy.
This framework proposes a context-editing framework that performs selective abstraction over entities that appear in both the context and the question, establishing symbolic abstraction as a highly cost-efficient solution for ensuring context fidelity in LLMs.
Rounak Sharma, Debabrata Mahapatra, S. Saini· Annual International ACM SIG...· 0 citations
Legal text generation and legal question-and-answer tasks impose stringent requirements on factual accuracy, evidence traceability and normative consistency. Conventional dynamic retrieval-augmented generation methods are difficult to directly adapt to legal task demands such as legal provision citation, terminology standardization and case evidence organization. Following the DRAGIN paradigm, this paper proposes LARIN (Lightweight Adapted Retrieval-Augmented Inference Network for Legal Issues), an adaptive retrieval-augmented reasoning framework tailored for legal scenarios. While maintaining the mainstream dynamic retrieval workflow, LARIN makes targeted optimizations in three key modules: retrieval triggering, query construction and evidence fusion. Specifically, LINDA identifies retrieval trigger points by comprehensively considering uncertainty, attention influence and semantic importance; JUDGE rewrites queries for legal terms and statutory expressions; MERF conducts relevance ranking, redundancy elimination and evidence fusion for legal provisions and case materials. In the CAIL2018 Chinese legal judgment prediction task, LARIN achieves a charge prediction exact-match accuracy of 0.3316, a micro-F1 score of 0.3837, and a precision of 0.3951, while the sentencing exact-match accuracy reaches 0.1575. Meanwhile, its average retrieval frequency is 1.81 and average token consumption stands at 373.15. Experimental results on CAIL2018 demonstrate that legal-oriented adaptation based on DRAGIN improves retrieval-augmented reasoning in this Chinese legal judgment prediction setting while maintaining low retrieval overhead. Cross-dataset robustness on additional legal QA or judgment benchmarks remains to be further validated.
Financial question answering over hybrid tabular and textual data may require multi-source reasoning and precise numerical computation. While large language models (LLMs) can generate intermediate reasoning steps, natural-language rationales remain prone to arithmetic errors, making them an unreliable supervision source for distillation. Building on programmatic distillation, we develop an approach that transfers reliable numerical reasoning from a large teacher model to a compact student using execution-verified Python programs instead of free-form textual rationales. It leverages gold derivations to guide teacher-side program synthesis and retains only programs that execute correctly and produce the gold answer, ensuring high-quality supervision. We further introduce an iterative recovery stage that revisits teacher-failed examples, enabling the student to recover and incorporate newly verified programs into training. Experiments on TAT-QA show that our framework is highly effective for hybrid financial reasoning. Our best 7B student achieves 87.00 EM / 87.18 F1 on the test set, substantially outperforming the 72B teacher (78.46 EM) as well as traditional and strong LLM-based baselines, including TAGOP and TAT-LLM. These results demonstrate that execution-verified programmatic distillation provides an effective and extensible framework for training smaller models to perform reliable numerical reasoning.
The results show that the quality of small ``base'' models can be greatly enhanced, and that reinforcement learning with verifiable rewards can be applied to NMT in the legal domain and surpasses the translation quality of supervised fine-tuning.
Aixiu An, Michael Jungo, Eloi Eynard et al.· 0 citations