A dynamic and self-corrective procedure with feedback-loop-based AI architecture with Large Language Models (LLMs) for near real-time validation and correction of generated queries and it is demonstrated that the proposed solution outperforms various training-free prompt injection defense methods.
Abstract
Text-to-Cypher generator systems translate natural language questions into Cypher queries, enabling intuitive interactions with graph databases such as Neo4j and Amazon Neptune. Despite recent advancements in LLM-based Cypher query generation, the vulnerabilities of the known methods—such as prompt injection attacks—are not discussed in detail. In this paper, we employ a robust Retrieval-Augmented Generation (RAG) architecture tailored specifically for text-to-Cypher tasks, leveraging dense vector retrieval to enhance query generation accuracy. We propose a dynamic and self-corrective procedure with feedback-loop-based AI architecture with Large Language Models (LLMs) for near real-time validation and correction of generated queries. We create a systematic procedure for generating datasets specifically designed to assess prompt injection robustness. Comprehensive evaluations are conducted using a diverse set of LLMs, including GPT-4o, DeepSeek R1, Claude 3.5 Sonnet and Qwen 2.5 Coder 32B Instruct. Our evaluation results indicate substantial improvements in resiliency against prompt injection attacks compared to various benchmarks. It is demonstrated that the proposed solution outperforms various training-free prompt injection defense methods.
Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.
Mukesh Lakshmi Sai Medikonda, Kalva Vishnu Teja, K. Greeshma et al.· International Conference Com...· 0 citations
CypherSmith is introduced, an instruction-tuning dataset over 12 × larger than prior public Text-to-Cypher datasets, spanning diverse domains to better support LLM fine-tuning and introducing a novel likelihood-based filtering technique to ensure high-quality Text-to-Cypher data.
Zeyu Zhang, Kexuan Sun, Zheng Tang et al.· Annual Meeting of the Associ...· 0 citations
Generator-Gate-Corrector (GGC), a framework for reliable LLM-based Text-to-SPARQL generation that enhances the accuracy, reliability, and efficiency of LLM-based text-to-SPARQL generation.
Ziyi Yang, Thanh-Son Nguyen, T. Nguyen et al.· 0 citations
This paper explores the usage paradigms of LLM text detectors for RAG and highlights key limitations of off-the-shelf or directly fine-tuned detectors.
Zhaoheng Huang, Yutao Zhu, Ji-Rong Wen et al.· Annual Meeting of the Associ...· 0 citations
Results demonstrate a 157% improvement in retrieval quality over a Naive-RAG baseline, with only 50 ms additional la tency, while Qwen2-VL-2B-Instruct achieved results comparable to cloud-based models in BERTScore, validate that open-source optimized SLMs, paired with advanced retrieval strategies, can provide competitive performance for document understanding without relying on cloud-based models.
This work conducts an empirical study of how irrelevant retrieved passages affect downstream generation, and proposes a lightweight, context-size classification module that dynamically predicts how much context is required based on query-specific needs.
Maya Iratni, M. Boughanem, T. Dkaki· Annual International ACM SIG...· 0 citations