Skip to content
Preprint

FuseRAG: Fusing Structure and Evidence into Inference for Cost-Efficient Graph-based RAG

Oct 2026 · 0 citations · 68 references
Computer Science

TL;DR

This work proposes FuseRAG, a cost-efficient graph-based RAG framework that shifts most retrieved evidence from explicit textual contexts to latent representations, and preserves graph structure through latent interactions, enables evidence reuse beyond exact token-prefix matching.

Abstract

Graph-based retrieval-augmented generation (RAG) exploits relationships among evidence to support multi-hop question answering. However, conveying retrieved graphs through non-unique linear textual contexts to Large Language Models (LLMs) leads to graph-structure degradation and inefficient KV-cache reuse. Meanwhile, graph-structure order optimization is already NP-hard, and the additional KV-cache reuse objective further complicates the search by introducing potentially conflicting orders. To tackle these issues, we propose FuseRAG, a cost-efficient graph-based RAG framework that shifts most retrieved evidence from explicit textual contexts to latent representations. Our main idea is to separate reusable evidence content from query-aware structural interactions in latent space, while retaining only a few explicit facts as semantic anchors. The resulting latent representations are injected into decoder self-attention, where they reshape attention over evidence and thereby influence hidden states and next-token probabilities. This design preserves graph structure through latent interactions, enables evidence reuse beyond exact token-prefix matching. Experiments on three multi-hop question-answering datasets with Llama-3-8B and Qwen3.5-9B show that FuseRAG improves answer quality while achieving up to 2.7x and 2.2x TTFT speedups over the fastest graph-based baseline, respectively.

View source

Similar papers

Open access Sep 2026

PoP-RAG: Holistic Query Planning for GraphRAG

PoP-RAG is proposed, which features a passage-on-edge (PoE) graph that links the passages with graph edges to resolve the one-to-many entity-passage mapping problem and enables targeted retrieval for each sub-query and ensures transparent response derivation logic, thus enhancing explainability.

Xin-Tong Hu, Qi-Ming Zeng, Yu-Hao Lin et al. · 1 citation
Preprint Oct 2026

Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG

Graph-based retrieval-augmented generation (GraphRAG) answers multi-hop questions by organizing a corpus into a graph and retrieving along its structure. Existing retrieval methods face two limitations. Diffusion- and similarity-based methods expand retrieval along fixed edge weights or by similarity to the question, w...

Jia-Te Liu, Liu-Yi Chen, Xiao-Fei Yuan et al. · 0 citations
Sep 2026

Beyond Similarity through Zero-Token Geometric Graphs for Multi-Hop RAG

Results show that geometric structure can support efficient multi-hop evidence discovery without LLM-based graph construction, and introduce Geometric Gain Graph RAG, a document-only framework whose offline graph construction uses no LLM calls or generated tokens.

Ze-Liang Li, Xiao-Fen Xing, K. Guo et al. · 0 citations
Open access Oct 2026

High-Information-Density Knowledge Graph Retrieval Augmentation for Large Language Models

Retrieval-Augmented Generation (RAG) mitigates hallucination and knowledge staleness in large language models (LLMs) by injecting external knowledge into the context window, yet its effectiveness is rigidly constrained by attention dilution and inference cost. Existing knowledge-graph-enhanced methods follow a sequenti...

Zi-Hao Zhou, Gang Yin, Ze-Jian Lu et al. · 0 citations

Corpus-Guided Dual-Path Propagation for Graph Retrieval-Augmented Generation

Graph-based retrieval-augmented generation supports multi-hop retrieval by organizing corpus information into graphs. However, existing relation-free graph retrieval methods rely primarily on query-sentence similarity to search for evidence. This can exclude useful bridging evidence with low query similarity and activa...

Ba-Xian Liu, Tong Wei · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.