A Deterministic Procedure-Aware Bilingual Retrieval-Augmented Generation Framework for Trustworthy High-Stakes AI Systems

Abstract

The advent of large language models and Retrieval-Augmented Generation (RAG) models has greatly enhanced intelligent information systems. This has resulted in the development of context-aware and knowledge-grounded response generation. This has been highly beneficial in the context of religious advisory systems, which require precision, correctness, and knowledge grounding. For Islamic rituals like Hajj and Umrah, the user needs precise and accurate procedures to follow, which must adhere to specific sequences and knowledge grounding. However, the existing models have many limitations in this context, like hallucinations, a lack of procedural knowledge, bilingual inconsistencies, and an inability to incorporate safety constraints. This has made these models unsuitable for contexts in which incorrect responses can have serious implications. Therefore, in this context, this paper proposes a Deterministic Procedure-Aware bilingual Retrieval-Augmented Generation (DPAM-RAG) model, which can be highly beneficial in designing religious advisory systems. The proposed model can be highly beneficial in designing religious advisory systems. The proposed model integrates dataset modeling, procedure-aware chunking, bilingual alignment, and deterministic transformer-based response generation. Additionally, a confidence-based refusal strategy has been proposed to avoid the generation of responses that can be considered incorrect or out of context. The proposed model has been tested through an extensive experimental setup, which includes multiple transformer models like GPT, LLaMA-2, Mistral, MPT, and BLOOMZ. The experimental results have shown promising outcomes, which can be considered highly beneficial in designing trustworthy AI models.