KGCaRe is proposed, a hybrid approach that combines neural retrieval with symbolic reasoning over LLM-generated KGs that consistently outperforms existing baselines, including Vanilla LLM, Code Prompt, Text Prompt, Think-on-Graph, Vanilla RAG, and HybridContextQA.
Abstract
Answering complex conditional questions using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) remains a challenge, particularly in domain-specific contexts where general-purpose LLMs and RAG tend to underperform. We hypothesize that augmenting RAG with unstructured and structured knowledge, extracted from both documents and knowledge graphs (KGs), can improve reasoning and answer accuracy for such tasks. To test this, we propose KGCaRe, a hybrid approach that combines neural retrieval with symbolic reasoning over LLM-generated KGs. KGCaRe constructs a KG from documents using a multi-prompt extraction strategy and stores it in a graph database. Simultaneously, the documents are embedded into a vector store to enable neural retrieval. KGCaRe performs innovative iterative graph traversal guided by the LLM to extract relevant triples, prune irrelevant information, and uses additional clue entities to traverse the graph again if the initial traversal does not provide satisfactory context to generate the answer. The relevant triples extracted from the KG in path form, along with semantically retrieved text passages, are then fed into custom KGCaRe prompts to generate answers to the complex conditional questions with explanations. We evaluate KGCaRe on two complex conditional QA datasets. Our results on these datasets show that KGCaRe consistently outperforms existing baselines, including Vanilla LLM, Code Prompt, Text Prompt, Think-on-Graph, Vanilla RAG, and HybridContextQA, across multiple LLMs such as Mistral, Mixtral, GPT-3.5, and GPT-4o. We publicly release the software pipeline that we developed to implement the proposed KGCaRe approach.
Large Language Models (LLMs) show considerable potential for materials-science question answering. However, LLM responses may still be affected by unsupported parametric associations, while dense Retrieval-Augmented Generation (RAG) can fragment relational evidence across text chunks. Moreover, general graph-based retrieval does not necessarily preserve the hierarchical relations and factual attributes required to resolve implicit material constraints. To address these limitations, we propose MCTD-KG, a multi-source heterogeneous knowledge graph integrated with a Knowledge-Enhanced RAG framework for complex material question answering. MCTD-KG adopts a Classification–Term–Data ontology to connect disciplinary taxonomies, domain concepts, semantic relations, and empirical records from toolbooks and the scientific literature. Through LLM-assisted knowledge extraction, entity normalization, and multi-source integration, the resulting graph contains more than 530,000 entities across three layers, including 61,768 text-extracted Term-layer entities. During inference, Dual-Channel Retrieval jointly retrieves query-relevant relational paths and associated material attributes, while an explicit semantic filtering stage screens candidate evidence against the query constraints. Evaluation on an expert-validated benchmark of 1577 questions shows that the proposed framework achieves an overall accuracy of 68.48%, compared with 17.40% for the zero-shot Pure LLM, 24.79% for the best Vanilla RAG setting, and 44.96% for GraphRAG. It also achieves 45.22% accuracy on four-hop questions, compared with 39.49% for GraphRAG. These results indicate that integrating multi-source domain knowledge with relation-preserved retrieval and attribute-supported filtering provides more focused and inspectable evidence, thereby supporting more accurate complex material question answering.
Peize Li, Xi Guo, Nan Yin et al.· Electronics· 0 citations
Knowledge graph question answering (KGQA) is a key task for evaluating KG-augmented Large Language Models (LLMs), and complex KGQA that requires multi-hop reasoning is especially challenging. Solving a complex query involves two coupled phases: candidate retrieval, which locates answer candidates over the KG, and constraint handling, which filters these candidates against the query constraints. Faithful reasoning requires grounding both phases in the KG. However, existing agent-based methods ground candidate retrieval through entity-centric exploration, while leaving constraint handling to the LLM's internal knowledge, which leads to two critical limitations. (1) Unreliable entity pruning: entity-centric exploration uses entities as search units and must prune them to a fixed-size subset at each hop. Because entity information in KGs is often incomplete and a fixed-size subset cannot retain all valid entities, such pruning inevitably drops valid entities and ultimately leads to wrong answers. (2) Ungrounded constraint handling: query constraints are resolved from the LLM's internal knowledge rather than the KG, leaving the final answers unverifiable and prone to hallucination. To address these limitations, this paper introduces a relation-centric exploration paradigm, which uses relations rather than entities as search units and thus avoids unreliable entity pruning. Built on this paradigm, this paper proposes Compositional Chain-of-Relations (CCoR), a simple and effective framework that grounds both phases in the KG with two relation chains: a main chain for candidate retrieval and a constraint chain that verifies query constraints through explicit KG exploration. Experiments on four KGQA benchmarks show that CCoR consistently improves accuracy, faithfulness, and efficiency over strong baselines, with more pronounced gains on complex queries.
Large language models are increasingly used for knowledge graph question answering (KGQA), but can fail to correctly ground answers in the underlying graph. Current approaches to LLM-based KGQA either rely on full semantic parsing into executable queries such as SPARQL, which is brittle in practice due to complex schemas or incompleteness of real-world KGs, or on LLM-reasoning and answer generation over KGs, which can be more robust but lacks formal guarantees. In this work, we study a complementary setting in which \emph{candidate} answers are generated by an LLM-based system and subsequently verified using lightweight symbolic constraints derived from the question. We introduce \emph{Constrained Entity Selection under Partial Knowledge (CES-PK)}, a problem formulation that focuses on eliminating invalid answers and providing symbolic support for valid ones without requiring construction of executable logical forms. To account for incomplete KGs, we employ a three-valued constraint semantics (\emph{satisfied, violated, unknown}) that avoids incorrect rejections under open-world assumptions. To demonstrate the effects of our method, we instantiate this framework over the Hetionet biomedical knowledge graph and evaluate the impact of type, relation, and exclusion constraints. Experiments show that precision improves by filtering invalid candidates, while recall is preserved due to retaining candidates whose constraints are not explicitly violated. Satisfied constraints provide additional positive symbolic evidence to rank remaining candidates.
Emanuel Kitzelmann· Deutsche Jahrestagung für Kü...· 0 citations
SSR, a Structured Subgraph Retrieval framework for TKGQA with Large Language Models, demonstrates that SSR consistently outperforms strong baselines by a clear margin, achieving state-of-the-art performance.
Ying Zhang, Li Zhang, Wenya Guo et al.· Annual International ACM SIG...· 0 citations
RA-QGQA is presented, which recasts triple verification as a question-driven, corpus-grounded task, and demonstrates RA-QGQA as an interactive web system in which users import a KG and its source corpus, verify all triples in a single pass, and inspect the passages that justify its verdict.
Siyang Liu, Hong Duc Nguyen, Yunmiao Li et al.· Proceedings of the 2026 ACM...· 0 citations
Retrieval-augmented generation (RAG) improves question answering by grounding large language models (LLMs) in external knowledge such as text corpora. However, its reasoning process remains largely opaque: intermediate reasoning steps are difficult to verify and cannot be reliably attributed to specific evidence. Moreover, missing user-specific context is rarely detected systematically, often leading to incomplete or incorrect output. We propose NeSy-RAG, a modular neuro-symbolic RAG framework that synthesizes attributable Prolog modules from retrieved text chunks. For each chunk, the system generates semantically meaningful predicates that encode Boolean claims, which may depend on user facts. Using joint natural language-code embeddings, predicates are retrieved and composed into Prolog queries. To address incomplete user context, we introduce a symbolic knowledge-gap detection mechanism that identifies missing user facts whose truth values affect the query outcome and automatically triggers follow-up interactions. Executing the resulting Prolog queries yields deterministic answers together with transparent execution traces that link each reasoning step to its originating source. On the ShARC benchmark, without domain-specific training, NeSy-RAG achieves 61.1% accuracy, outperforming a same-model RAG baseline that achieves 42.8% accuracy.