Graph and multimodal extensions to retrieval-augmented generation (RAG) are often evaluated end to end, making it difficult to isolate whether gains arise from retrieval, prompt-side context, visual access, generator capability, or benchmark construction. We present a stage- and evidence-controlled evaluation across five RAG configurations, four multimodal generators, and three document corpora. The same LLM-extracted knowledge graph is used either after retrieval as provenance-constrained triple injection (+KG) or during retrieval as entity-bridged passage expansion (+KGret). Prompt-side graph injection yields no consistent accuracy improvement and generally reduces faithfulness. In contrast, +KGret increases gold-evidence completeness from 0.22 to 0.46 on HotpotQA bridge questions and from 0.50 to 0.72 on SPIQA cross-paper questions, improving accuracy for every generator on both evidence-deficient sets while having little effect on retrieval-complete controls. For visual question answering, matched caption-answerable and verified pixel-only protocols show that apparent multimodal gains are sensitive to textual leakage. Programmatic checks reveal answer recoverability from captions, corpus text, and model responses generated without complete gold evidence. Accuracy on incomplete-evidence questions reaches 0.35--0.71 on widely disseminated corpora, compared with 0 on PubLayNet, indicating that raw accuracy can overstate retrieval-attributable performance. These results show that graph augmentation is most effective when it changes retrieval under evidence deficits, while multimodal evaluation requires explicit verification that answers are unavailable through text.
Retrieval-augmented generation (RAG) systems commonly operate on text extracted from documents, potentially losing information contained in figures, tables, layout, and relationships distributed across passages. We present an explainable multimodal graph-RAG architecture that augments a text-only baseline with LLM-extracted subject--relation--object triples and CLIP-based retrieval of figures and tables. The three evidence sources are retrieved independently and fused only at generation time, allowing the effects of graph evidence, visual evidence, and generator choice to be evaluated separately. We conduct a controlled four-way ablation on 1,000 PubLayNet pages across single-passage, multi-hop, and figure questions using two closed-weight and two open-weight multimodal generators. We also compare matched caption-answerable and pixel-only figure question sets to distinguish caption recovery from genuine visual question answering. Knowledge-graph augmentation provides no reliable accuracy improvement across generators or question types in this corpus. On pixel-only questions, text-only systems achieve zero accuracy, while multimodal systems reach 0.057--0.114 and are constrained by both image retrieval (Recall@3 = 0.371) and the generators'ability to interpret dense scientific figures and tables. Caption-derived questions substantially overestimate text-only visual question-answering capability. Processing the same image also produces an elevenfold variation in input tokens across generators, showing that image tokenisation can dominate deployment cost. Source code, experiment configurations, and evaluation resources are publicly available at https://github.com/sokistar24/multimodal-graph-rag. These findings show that the value of multimodal and graph augmentation depends on retrieval design, corpus structure, benchmark construction, and generator efficiency.
