Skip to content
Log in
Sponsored Design Multi-Agent AI Systems Using MCP and A2A: Engineer your own Pyt Design Multi-Agent AI Systems Using MCP and A2A: Engineer your own Pyt
Preprint

EduGuard: A Safe RAG-Based LLM Tutor for Programming Education

S. Hossain Ruksat Khan Shayoni M. F. Mridha Jungpil Shin
Jul 2026 · 0 citations · 35 references
Computer Science

TL;DR

E EduGuard, a safe retrieval-augmented generation (RAG) tutoring framework for introductory programming, is presented and compared against strong baselines, suggesting safe GenAI tutoring requires not only retrieval or strong prompting, but explicit pedagogical control, evidence verification, and deployment safeguards.

Abstract

Generative AI (GenAI) is increasingly used by students for programming explanation, debugging, and assignment support. Yet unrestricted large language model (LLM) tutors can hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence. This paper presents EduGuard, a safe retrieval-augmented generation (RAG) tutoring framework for introductory programming. EduGuard integrates query understanding, instructor-approved course retrieval, pedagogical strategy selection, rubric-aware generation, claim-level verification, and overreliance control. To make evaluation provenance explicit, we construct BILearn-CS, a 600-query instructor-authored, TA-validated benchmark spanning concept questions, debugging cases, misconceptions, assignment-support requests, code-mixed Bangla-English queries, and adversarial direct-answer prompts. Moving beyond a synthetic-only benchmark, we further evaluate on a 150-query public CS50-style course-forum set and run a small controlled pilot with 10 undergraduates using a counterbalanced pre-test/post-test design. Using Meta-Llama-3.1-8B-Instruct as the primary generator, hybrid FAISS/BM25 retrieval, and DeBERTa-v3-large-MNLI as an architecturally separate verifier, EduGuard is compared against strong baselines: GPT-4o-mini Tutor, Llama Socratic Tutor, LPITutor-style RAG, RAG with rubric prompting, and RAG with same-model self-checking. On BILearn-CS, EduGuard attains the best correctness (90.1%), grounding (89.4%), and rubric alignment (90.8%), with the lowest hallucination (4.9%) and direct-answer leakage (9.8%). In the pilot, it raises immediate post-test accuracy from 68.4% to 81.2% and cuts overreliance from 38.0% to 17.0% relative to GPT-4o-mini Tutor. These results suggest safe GenAI tutoring requires not only retrieval or strong prompting, but explicit pedagogical control, evidence verification, and deployment safeguards.

arXiv Semantic Scholar
View source

Similar papers

Preprint Jul 2026

CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

CSTutorBench is introduced, a benchmark for evaluating language models as CS tutors in VEX VR, a block-based robotics environment, and preliminary findings reveal that models perform well on surface-level criteria such as vocabulary and tone but struggle with deeper pedagogical behaviors.

H. Chad Lane, Bryson Kageler · 0 citations
Conference Jul 2026

Designing Reliable LLM Systems for AI-Assisted Engineering Education

Large Language Models (LLMs) are increasingly used in AI-assisted engineering education, enterprise learning, technical knowledge support, and software-development workflows. However, static retrieval-augmented generation (RAG) pipelines retrieve external context for every query, which can increase latency, token cost, and irrelevant-context exposure, while direct generation can produce unsupported or stale answers for policy-sensitive, documentation-dependent, or multisource questions. This paper presents a production-oriented adaptive retrieval framework that treats retrieval as a conditional reliability decision rather than a default execution step. The framework combines query embeddings, similarity-based query memory, feedback-confirmed routing labels, comparison/fallback handling, and observability logs to decide whether a query should be answered directly, routed through retrieval, or escalated for evidence comparison. The system was evaluated in a production-similar controlled test bed using a 1,300-scenario validation workload and public benchmark validation using HotpotQA and RAGTruth. In the 1,300-scenario evaluation, the proposed method achieved 91.31% routing accuracy, 85.00% grounded-answer rate, 85.38% unsupported-answer detection, and 79.85% fallback correctness, while reducing mean latency compared with always-on Static RAG. Paired statistical testing showed significant improvements over LLM-only, Static RAG, and Threshold RAG baselines for routing, grounding, unsupported-answer detection, fallback behavior, and latency. The paper also discusses deployment considerations including query-memory growth, stale no-retrieval labels, knowledge-base version tracking, privacy-preserving redaction, and instructor oversight for responsible educational use.

Ramesh Venkatachalapathy · 0 citations
Preprint Aug 2026

EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners

EduClaw-Bench is introduced, a benchmark that places an agent tutor in a continuous 30-day relationship with a simulated learner grounded in knowledge tracing (KT), whose knowledge-concept mastery, from a KT model trained on real-student data, drives its answers and is probed for learning gain across 55 scenarios.

Unggi Lee, Sookbun Lee, Yeil Jeong et al. · 0 citations
Preprint Aug 2026

Auditable Release Control for Pedagogical Leakage in LLM Tutors

These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains, not universal semantic safety or learning gains.

Nizam Kadir · 0 citations
Open access Aug 2026

Lightweight Llama Models with Experts’ Curated RAG for Electrical Engineering Education: An Exploratory Comparison

The results show that the proposed RAG agent substantially improves the lightweight base model and produces transparent, syllabus-grounded answers that experts rated as correct and concise, while GPT-4.5 retains an edge on longer, multistep, and topology-intensive tasks.

André Rocha, Paulo Oliveira, João Ferreira et al. · 0 citations
Book Open access Jul 2026

A Good Rubber Duck Does Not Quack: Designing Socratic Scaffolding in AI Tutors

Socratic AI, a VS Code-integrated tutor that addresses this through pedagogically-grounded Socratic dialogue constrained to withhold direct solutions is presented, and evidence that stateful tracking enables adaptive Socratic dialogue that scaffolds productive struggle rather than short-circuiting learning is contributed.

Ayush Thonge, Aalok Thakkar · 0 citations