Managing diabetic kidney disease (DKD) is inherently complex, requiring clinicians to synthesize patient history, fluctuating biomarkers, and evolving treatment guidelines. While large language models (LLMs) show promise in medical decision support, their clinical adoption is hindered by factual inaccuracies and a lack of specific reasoning required for individualized patient management. To address this, we developed a hierarchical multi-agent system that integrates a locally deployed retrieval-augmented generation (RAG) framework with a cloud-based advanced reasoning engine, grounding responses in a curated corpus of clinical guidelines. We conducted a multi-center retrospective validation using 267 patient cases. The system's performance was evaluated against baseline models through a blinded review by twelve independent physicians across clinical dimensions including accuracy, safety, and factuality. Our evaluation reveals that the RAG-enhanced system significantly outperforms unaugmented models in providing accurate, guideline-compliant recommendations. Notably, it substantially reduced safety-critical errors, particularly in identifying medication contraindications related to renal function stages, while achieving high inter-rater reliability. This study demonstrates that anchoring LLMs with authoritative knowledge effectively mitigates hallucination risks and enhances clinical reliability. The proposed framework functions as a reliable on-demand assistant for DKD management, providing guideline-grounded decision support for primary care providers.
Xuan Tao, Lan Tian, Chenhao Fang et al.· npj Digital Medicine· 0 citations
On-policy self-distillation (OPSD), where a single model acts as both student and teacher with different contexts, has shown promise in verifiable domains like math, where hard privileged information (PI) in the form of ground-truth answers structurally constrains valid continuations. We extend OPSD to open-ended generation using soft PI in the form of rubrics that guide preferences but admit many valid responses. Rubrics have served as scalar rewards for reinforcement learning (RL); we show that they provide substantially richer signal as dense PI for distillation, and contrary to intuition, soft rubric PI provides a larger and more effective training signal on student roll-outs than hard reference completion PI in this regime. A reference completion is one point in a set of valid responses, so distilling towards it over-constrains the student, while rubrics specify the preference structure shared across the set of valid responses. We show the effectiveness of using rubrics as PI for open-ended generation across Qwen and Llama model families and show that it outperforms rubric-as-reward (RaR) RL using HealthBench, a benchmark that grades open-ended health responses against physician-created rubrics, providing dense token-level supervision for open-ended tasks; RuPI beats RaR RL by up to +0.10 absolute score and, under matched recipe and KL direction, beats reference-PI by +0.034 to +0.079 absolute score across three models. We further show that these findings generalize to training on the RubricHub Science corpus and evaluating on ResearchQA: soft rubric PI outperforms both reference-PI distillation and RaR RL (66.6% vs. 64.2% and 57.6%).