GCA-KBQA: A Step-Wise Logical Form Generation Approach for KBQA with Knowledge-Assisted Calibration

Abstract

Knowledge base question answering (KBQA) aims to answer natural language questions using large-scale knowledge bases (KBs). Among various KBQA approaches, semantic parsing-based (SP-based) methods have demonstrated strong effectiveness by generating concise logical forms (LFs) that capture complex subgraph structures and semantic information. Recent research suggests that integrating large language models (LLMs) with SP can achieve significant improvements in the performance and efficiency of KBQA by facilitating the direct generation of LFs with minimal retrieval. However, generating complete LFs with LLMs continues to pose a challenge due to the complexity of the required graph structures and constraints, leading to the significant issue of non-executability. To address these challenges, we propose GCA-KBQA, a step-wise fine-tuned LLM-based framework that employs hop-wise generation, knowledge-assisted calibration, and path-level assembly to construct complete LFs for KBQA. Specifically, we decompose the complex SP process into manageable steps: first, we iteratively generate LFs for each topic entity one hop at a time using a fine-tuned LLM, leveraging KB knowledge to calibrate intermediate outputs and mitigate error propagation. Subsequently, we guide the LLM in assembling path-level LFs from different topic entities, resulting in optimized final LF. We evaluate the proposed method on four KBQA benchmarks spanning two distinct KBs, demonstrating its superior performance compared to state-of-the-art baselines. The code is available at https://github.com/pvfeldt/GCA-KBQA.