Clinical documentation quality in routine care with AI scribe support: a large‑scale Spanish analysis

Abstract

Clinical documentation is essential for care quality, safety, and coordination, yet the growing burden of electronic health record (EHR) tasks contributes to clinician burnout and may compromise documentation quality. Ambient artificial intelligence (AI) scribe systems have emerged as a potential solution, but their impact on documentation quality and readability remains underexplored. Therefore, we aimed to evaluate the effect of an AI-assisted scribe system (Scribe) on the semantic agreement, readability, and structured quality of outpatient clinical documentation across multiple specialties in a large Spanish healthcare network, within the framework of value-based healthcare (VBHC). Thus, we conducted a retrospective, multi-method study of over 2.3 million outpatient consultations across 45 specialties (Oct 2024–Dec 2025). Documentation with and without Scribe was compared using automated metrics (semantic agreement, INFLESZ and Perspicuity) and an 18-item structured quality rubric. Semantic agreement was evaluated across all 45 specialties, whereas report quality was assessed in a stratified sample of 300 reports from 15 specialties. A manual audit evaluated completeness of key safety elements (past medical history, allergies), and statistical analyses included descriptive metrics, Welch and paired t-tests, and Hedges’ g. Notably, Scribe achieved a weighted mean semantic agreement of 89.3% (SD 2.1). Readability improved significantly: INFLESZ scores increased from 72.05 to 77.61 ( p  < 0.001), and Perspicuity scores from 76.19 to 81.49 ( p  < 0.001). Structured quality scores rose from 5.79 (95% CI 5.60–5.99) to 8.02 (95% CI 7.87–8.17), with a large effect size (Hedges’ g = 2.06; p  < 0.001). A manual audit showed significant improvements in documentation of past medical history (7.75 vs 1.75; p  < 0.001; dz = 1.10) and allergy status (9.75 vs 7.75; p  = 0.010; g = 0.43). Altogether, AI-assisted scribe technology was associated with improvements in documentation quality, readability, and safety-related documentation elements. These improvements align with VBHC principles by supporting better patient outcomes, safer care, and more efficient use of clinical resources through high-quality documentation.