HIEVI-RAG is introduced, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding that significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.
Abstract
Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HIEVI-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HIEVI-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EVIAGENT, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the strongest reported baseline by an average of 8.05% in accuracy.
This work introduces MMHRAG, a novel Multi-Modal Hierarchical Retrieval-Augmented Generation framework that achieves cross-modal interaction on DocQA for the first time, and designs a Summarizing Agent to resolve logical conflicts, information redundancy, and granularity discrepancies among retrieved cross-modal evidence.
Jiayuan Wang, Jie Lian, Fu Zhao et al.· Proceedings of the 32nd ACM...· 0 citations
Retrieval-augmented generation (RAG) has become a standard paradigm for knowledge-intensive question answering by grounding large language models (LLMs) in external evidence. However, open-domain multi-hop question answering (QA) remains challenging for two reasons. First, evidence dispersion across documents and non-contiguous spans means that critical bridge evidence can be weakly related to query and is easy to miss. Second, semantic-resolution mismatch complicates retrieval: coarser retrieval views offer better global coherence but may obscure the exact bridging detail, while finer-grained views highlight specific mentions but may omit the context needed to reveal the relation. In this paper, we propose MCoRe, a multi-entry complementary retrieval framework with reflection-guided iteration for multi-hop QA. To mitigate the semantic-resolution mismatch, MCoRe enables multi-entry complementary retrieval by indexing entry units at multiple semantic resolutions (entities, sentences, and summaries) with explicit links to chunk evidence, mapping all hits back to chunks, and fusing cross-resolution hits via chunk-level voting to form a compact evidence set for answer generation. To cope with evidence dispersion, MCoRe performs reflection-guided iteration: when evidence is insufficient, it identifies the missing bridge cue and issues a gap-focused follow-up query to recover it. Empirical results demonstrate the effectiveness of MCoRe, which consistently outperforms state-of-the-art baselines by 6.77 EM points and 8.79 F1 points averaged over three multi-hop QA benchmarks, with gains of up to 12.70 EM and 14.06 F1 points on 2Wiki.
Juxiang Zeng, Zhuohui Gao, Zhe Hou et al.· Proceedings of the 32nd ACM...· 0 citations
Graph-based and multimodal retrieval frameworks provide a strong foundation for long-document question answering, but single-pass retrieval can remain brittle when queries are ambiguous, multi-step, or misaligned with the indexed evidence. We present Agentic-RAG, a structure-aware retrieval-augmented generation framework that combines a MinerU-LightRAG-based document processing and graph retrieval pipeline with an LLM-based agentic query-control layer. The base pipeline supports structure-aware parsing and VLM-based captioning for textual, visual, and tabular evidence, while the agentic layer performs query planning, LLM-based evidence reranking, answer generation, groundedness checking, relevance checking, and query reformulation. The framework does not modify the underlying graph construction or indexing mechanism; instead, it improves retrieval control by guiding the base retriever toward evidence that better matches the user's information need. Experiments on HotpotQA and ASQA show that Agentic-RAG improves context precision on HotpotQA from 0.1682 to 0.2462 and substantially improves context precision and context recall on ASQA from 0.5045/0.2783 to 0.6522/0.4783. The results indicate that agentic query control is especially useful for ambiguity-heavy long-form reasoning, while sparse multi-hop evidence chaining remains a bottleneck for future work.
D. Lam, Gia Hien Tran, Tien-Dung Do· 2026 11th International Conf...· 0 citations
Results demonstrate a 157% improvement in retrieval quality over a Naive-RAG baseline, with only 50 ms additional la tency, while Qwen2-VL-2B-Instruct achieved results comparable to cloud-based models in BERTScore, validate that open-source optimized SLMs, paired with advanced retrieval strategies, can provide competitive performance for document understanding without relying on cloud-based models.
Retrieval-Augmented Generation (RAG) has established itself as a compelling strategy for grounding large language model outputs in documentary evidence. However, production deployments continue to rely almost exclusively on homogeneous text corpora, even as enterprise repositories grow increasingly heterogeneous blending technical schematics, radiological images, annotated diagrams, and unstructured prose within the same archival system. This mismatch between system design and data reality motivates the present work. We propose a Multimodal RAG framework that unifies text and image retrieval through four tightly coupled components: a dual-stream embedding engine, a learned four-class AI query router, an adaptive confidence threshold, and a session-aware context store. Text is encoded with the allmpnet-base-v2 Sentence Transformer; images are embedded in the same 768-dimensional space via a domain-adapted CLIP ViT-L/14 model fine-tuned on approximately 120,000 technical and clinical text-image pairs. Concatenating 768-d vectors yields a 1,536-d composite query that drives a single approximate nearest-neighbor (ANN) search simultaneously across both modalities. A fine-tuned DistilBERT router assigns each query to one of four retrieval pathways text-only, image-only, hybrid, or conversational at 91.3% accuracy. Experiments were run on three enterprise corpora (engineering manuals, clinical case summaries, and legal paperwork) with the results being a mean F1@5 of 0.90, an increase of 22% over a dense text-only baseline, and a reduction of 49% in the number of hallucinations. The median first-token latency is 1.34s, which meets the interactive-use target deployment-contexts. These results show that it is possible to implement modality-aware retrieval in a simple and realistic setting with real-world organizational constraints and that the results are reproducible.
E.Vijayakumar, Ganesh A· 2026 4th International Confe...· 0 citations
Document question answering over scanned pages requires two coupled abilities: (i) canonicalizing complex layouts into a faithful textual structure, and (ii) selecting and reasoning over query-relevant evidence from that structure. Most existing pipelines decouple OCR from retrieval-augmented reasoning and optimize OCR for global reconstruction, which often misaligns with evidence needs and causes brittle grounding in multi-page settings. We propose UniDocVLM, an end-to-end framework that unifies OCR and visual RAG within a single vision-language model: the model first generates a structured parse of retrieved pages, then activates question-relevant evidence from the parse to support grounded reasoning and answering. To train UniDocVLM under heterogeneous supervision, we introduce a unified JR-GRPO reinforcement learning recipe with lightweight, verifiable rewards, including format, layout-aware OCR, evidence-consistency, and answer-correctness signals, and route them to the corresponding parts of the output to improve credit assignment and reduce interference. Experiments on multi-page document QA benchmarks show that UniDocVLM yields more reliable evidence grounding and improves downstream accuracy under complex layouts.
Zongsheng Cao, Anran Liu, Jun Xie et al.· Proceedings of the 32nd ACM...· 0 citations