Rhetorical-Role-Aware Retrieval-Augmented Generation for Legal Question Answering over Indian Supreme Court Judgments

Leveraging Retrieval-Augmented Generation to Support Legal Question Answering: Evidence from Indonesian Human Trafficking Court Decisions

The exponential growth of court decisions in Indonesia has posed a crucial challenge for legal practitioners in obtaining relevant information. Conventional search systems fail to capture the in-depth legal context, while Large Language Models (LLMs) are prone to producing hallucinations that can mislead legal reasoning. This study proposes and tests the implementation of Retrieval-Augmented Generation (RAG) to support Legal Question Answering LLM-based Quality Assurance (LQA) to improve factual accuracy. This study used 408 Indonesian court decisions related to criminal cases. Human trafficking data collected and standardized from 143 district courts. The RAG framework is designed in three stages: indexing, search, and incremental generation. We evaluated three Open-source LLM models: Gemma, LLaMA, and Qwen. Three models are also combined with two retrieval methods: BM25 (lexical) and Dense (semantics). Experimental results show that Qwen 3, especially when combined with BM25 RAG, consistently produces the highest overall answer quality across all evaluation metrics (ROUGE and BLEU). The BM25 method is significantly more effective than dense retrieval. Due to the highly standardized nature of court decision documents, Qwen demonstrated peak performance on structured information categories such as “identitas_terdakwa”, achieving a ROUGE-L score of 0.899. The primary contribution of this study is demonstrating that RAG significantly reduces the risk of LLM hallucinations in a legal context. However, error analysis suggests that future improvements should focus on strengthening generation controls to address these issues: unsupported generation and remaining model-rejection behavior.

Evaluating RAG approaches for Question Answering

CoAL-RAG: A Complexity-Aware Legal Retrieval-Augmented Generation Method

Legal consultation questions exhibit multi-level complexity. A single retrieval strategy often leads to over-reasoning for simple questions and poor interpretability for complex ones, making it difficult to meet the requirements for both answer quality and efficiency in high-risk scenarios. To address this issue, this paper proposes CoAL-RAG, a complexity-aware legal retrieval-augmented generation method, which constructs a multi-dimensional evaluation mechanism based on ``question essence''and ``retrieval consistency''to enable adaptive routing of retrieval strategies. First, the reasoning demand is quantified according to the logical structure of the question. Then, the discrepancy between semantic retrieval and keyword retrieval is utilized to indirectly reflect problem complexity, thereby selecting the most appropriate retrieval strategy and dynamically filtering contextual information. Experimental results demonstrate that the proposed method significantly outperforms baseline models not only on Chinese legal benchmarks (SocialLawQA, LawBench) but also demonstrates strong cross-jurisdictional generalization on English datasets (LexGLUE, CaseHold). Specifically, on Chinese datasets, the BLEU score improves by 42.5\% and ROUGE-L reaches 3.6 times that of knowledge graph-based methods. On English benchmarks, CoAL-RAG maintains highly competitive accuracy, achieving an optimal balance between generation quality, deep logical reasoning, and system efficiency across different legal systems.

Bridging Legal Language Barriers Using Explainable AI: Outcome Prediction and Multilingual Knowledge based answer retrieval for Indian Law

This study presents an integrated legal AI platform that combines interpretable case outcome prediction with multilingual, retrieval-grounded legal question answering to improve access to Indian law. The work is motivated by the difficulty ordinary citizens face in understanding legal language, the scarcity of trustworthy guidance, and the need for tools that work across India’s major languages. To address this, the authors built two connected components: a prediction module for Supreme Court case outcomes and a question-answering module based on statutory retrieval and generation. For prediction, they compiled 26,688 Indian Supreme Court judgments from 1950 to 2024 and represented each case using TF-IDF text features, case-type encodings, and temporal metadata, then trained an interpretable logistic regression model. For legal QA, they indexed 21 Indian legal acts with sentence-transformer embeddings and FAISS, and used a locally hosted Mistral model to generate simplified answers grounded in retrieved legal passages. The system was designed for English, Hindi, and Tamil, with translation, speech input, speech output, and interactive visualizations to make legal information more accessible. The prediction model achieved 91.3% accuracy and 0.919 ROC-AUC, while confidence calibration showed a strong correlation between predicted and actual accuracy. In the QA module, the system reached 78.4% precision@5, 86% answer correctness, and only 7% hallucination, a substantial improvement over baseline generative approaches. User evaluation with 35 participants reported 4.05/5 overall satisfaction, with multilingual support and explainability among the most valued features. Overall, the study concludes that transparent machine learning, retrieval-augmented generation, and multilingual interfaces can work together to build a practical and trustworthy legal assistance system for Indian users.

An Optimization Framework for Retrieval Augmented Generation in Indonesian Educational Question Answering

A RAG optimization framework for Indonesian-language educational question answering using a Human-Computer Interaction learning corpus as a case study is developed and provides a procedure for selecting retrieval and generation settings for a given corpus.

An Intelligent PDF Question-Answering System; A Retrieval-Augmented Generation Approach

Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.