PERA-Tutor: A Personalized Explainable Retrieval-Augmented Tutor for Educational Question Answering

Abstract

Retrieval-augmented generation (RAG) is attractive for educational question answering because external evidence can improve provenance, while learner models can adapt explanations to prior knowledge and misconceptions. However, apparent gains can be overstated when synthetic benchmarks leak topic labels or make unanswerable queries trivially separable. This paper introduces PERA-Tutor, a personalized and explainable retrieval-augmented tutoring architecture, and reports a deterministic retrieval-stage evaluation that can be reproduced without a language model. The benchmark contains 240 curriculum chunks, 1,200 questions (900 answerable and 300 unanswerable), and 1,500 retrieval stress cases. With a BM25 first stage and a 25-document candidate pool, the prespecified pedagogical re-ranking point (semantic/topic/difficulty/misconception weights 0.55/0.20/0.15/0.10) increases Recall@5 from 0.550 to 0.624 versus semantic-only ranking (paired delta = 0.074, exact McNemar p = 1.09e-7) and increases Difficulty Fit@5 from 0.625 to 0.744 (delta = 0.119, Wilcoxon p = 2.18e-49). Yet a post-hoc no-topic reallocation yields only a 0.011 Recall@5 gain (p = 0.477), because every answerable template contains its gold topic name. Masking that surface form reduces PERA Recall@5 to 0.090. Separately, raw BM25 top scores distinguish answerable from unanswerable queries with AUC = 0.944 overall and 0.917 for in-corpus-subject negatives. A five-fold calibrated retrievalscore baseline (B0) correctly abstains on all 300 unanswerable queries while accepting 817 of 900 answerable queries. These findings support difficulty-aware ranking as a mechanism for pedagogical fit, but not a general claim of retrieval or hallucination superiority. They also establish two requirements for future end-to-end evaluation: leakage-resistant questions and verification baselines that outperform calibrated retrieval-score abstention.