LPDG-RAG: Leveraging Probabilistic Rewards and Dynamic Granularity for Retrieval- Augmented Generation

Abstract

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by incorporating external documents, but its effectiveness degrades when retrieved passages are irrelevant, incomplete, or inconsistent with parametric knowledge. We present LPDG-RAG, a post-training framework that extends reasoning-based knowledge refinement with three coordinated mechanisms. First, a Probability-Contrastive Reward Model (PCRM) derives continuous preference scores from Yes/No-renormalized label probabilities under mirrored response orders. Second, dynamic-noise Group Relative Policy Optimization (GRPO) exposes the policy to varying mixtures of supportive and noisy passages, allowing it to learn when retrieved evidence should be used or discounted. Third, Dynamic Granularity Knowledge Refinement (DGKR) generates query-dependent triples at the minimum density sufficient for correct answering. During inference, the GRPO policy produces a retrieval-conditioned reasoning trace, DGKR supplies structured evidence, and the policy synthesizes the final answer from both streams. Across three independently trained runs (seeds 42, 43, and 44), LPDG-RAG consistently outperforms RankCoT, the principal reasoning-based refinement baseline, with mean gains of 4.75 points on NQ, 3.91 on HotpotQA, 1.60 on TriviaQA, 4.59 on PopQA, 5.09 on ASQA, and 6.64 ROUGE-L on MS MARCO. Additional comparisons with Self-RAG- and CRAG-based adaptations, component ablations, group-size sensitivity, granularity analyses, and controlled retrieval corruption provide additional evidence about the proposed design.