Generator-Gate-Corrector (GGC), a framework for reliable LLM-based Text-to-SPARQL generation that enhances the accuracy, reliability, and efficiency of LLM-based text-to-SPARQL generation.
Abstract
Large language models (LLMs) have demonstrated strong capabilities in structured query generation, making them a natural choice for Text-to-SPARQL, which translates natural language questions into executable SPARQL queries over knowledge graphs. However, their initial outputs remain unreliable: generated queries may be executable yet semantically misaligned with input questions, leading to incorrect retrieval. To address this issue, we propose Generator-Gate-Corrector (GGC), a framework for reliable LLM-based Text-to-SPARQL generation. GGC first uses a Generator to produce an initial query, then applies a Gate to predict whether correction is needed, and finally invokes a Corrector only for selected high-risk queries. This selective correction mechanism avoids unnecessary modifications and reduces the risk of degrading originally correct queries. Experiments on MCQA show that GGC improves query-level accuracy from 90.23\% to 98.33\% while reducing inference overhead by 45\% compared with correcting all generated queries. Ablation studies show that the Gate is robust across thresholds and that Corrector training data composition affects correction effectiveness and stability. Overall, the results demonstrate that selective correction enhances the accuracy, reliability, and efficiency of LLM-based text-to-SPARQL generation.
As Large Language Models (LLMs) become foundational to next-generation Intelligent Information Systems, the bridge between natural language interfaces and structured database systems remains a critical bottleneck. While Text-to-SQL generation enables cooperative support for complex query formulation, ensuring the reliability of these generated queries at inference time is a central challenge. Conventional methods rely on coarse execution-based signals, which may limit their ability to capture the nuanced semantic alignment required for high-stakes database environments. In this work, we propose the use of Outcome Reward Models (ORMs) as a fine-grained, probabilistic feedback mechanism for test-time verification in Text-to-SQL tasks. We introduce GradeSQL, a framework for training task-specific ORMs that assign scalar utility scores to candidate SQL queries based on their semantic correctness and alignment with database schema. Our approach is evaluated on the BIRD and Spider benchmarks across multiple open-source LLM families. Experimental results demonstrate that ORM-based verification consistently outperforms traditional execution-based heuristics.
M. Tritto, G. Farano, Dario Di Palma et al.· Journal of Intelligence and...· 2 citations
Decisions are being made based on data, and the demands for easy-to-use interfaces that enable users to simply type in a question to a relational database without having to be a Structured Query Language (SQL) expert have increased. Although the development of Large Language Models (LLMs) has seen remarkable advancements in the past few years, and has led to the emergence of LLMs that have significantly improved the generation of SQL from natural language (Text-to-SQL)[1], there is a relative lack of systematic comparisons between the latest commercial LLMs that has been tested against a series of graded SQL complexity levels with both syntactic and execution-based metrics. This research aims to assess and contrast ChatGPT, Google Gemini, DeepSeek, and Microsoft Copilot for their accuracy and efficacy in creating SQL queries from natural language queries. In a quantitative experimental design, 100 natural language questions were given using a purpose-built benchmark, with six types of queries that included simple retrieval, filtering, joins, aggregation, GROUP BY and HAVING, and nested subqueries. The results of the models were assessed using the manually written and validated ground truth query, with the use of normalised Exact Match Accuracy (EMA) and Execution Accuracy (EXA). The best overall results (EMA 97%, EXA 97%) were obtained by ChatGPT. The Copilot scored EMA 56% and EXA 96%, DeepSeek EMA 55% and EXA 93% and Gemini the lowest with a score of EMA 36% and EXA 87%. The large margin between EMA and EXA for the models (+51 points, +40 points, +38 points) demonstrates the significant shortcomings of exact-match scoring when it comes to assessing real-world skills for producing valid but structurally different SQL. The study empirically demonstrates comparative behaviours of the current LLMs, and suggests the use of execution-aware evaluation in the studies and deployment of Text-to-SQL.
Bharath Kumar N, T L Manasa· International journal of com...· 0 citations
Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task-specific fine-tuning. However, existing LLM-based systems remain unreliable, often generating SQL queries that are invalid under the database schema, referencing non-existent tables, attributes, functions, or values. Such errors persist because interactions with the database management system (DBMS) are typically limited to error messages, leaving it in a largely passive role during query refinement. This paper proposes SafeQL,
a search-based refinement paradigm that redefines the role of the DBMS as an active guide in the refinement process.
Instead of regenerating entire queries after execution failure, SafeQL interprets DBMS feedback to incrementally repair only the erroneous components. Each refinement step is formulated as a guided search within a
safe query space
, where candidate queries are progressively validated through DBMS execution, thereby converging to an executable query and preventing repeated regeneration of errors. Experiments on the Bird and Spider benchmarks show that SafeQL significantly improves execution accuracy and efficiency compared to regeneration-based methods.
Geonho Lee, Min-Soo Kim· Proceedings of the VLDB Endo...· 0 citations
This work proposes MARS, a scalable knowledge graph question answering (KGQA) approach that requires no model fine-tuning, and performs a structured retrieval procedure that links question entities to the KG and iteratively retrieves relevant next-hop information.
Nikit Srivastava, Daniel Vollmers, René Speck et al.· 0 citations
This semantic path compilation (SPC) system against direct DDL-to-SQL generation on the ACME insurance benchmark supports an end-to-end systems result, not a causal claim that compilation alone produced the gain, because SPC receives governed semantic artifacts that the DDL baseline does not.
A dynamic and self-corrective procedure with feedback-loop-based AI architecture with Large Language Models (LLMs) for near real-time validation and correction of generated queries and it is demonstrated that the proposed solution outperforms various training-free prompt injection defense methods.