Do Open-Weight LLMs Respect Minimum-Necessary Access in Text-to-SQL? An Automated Audit on EHR Benchmarks

Abstract

Large language models are increasingly deployed as natural-language interfaces to clinical databases, yet their evaluation has focused almost exclusively on query correctness rather than access minimality. Data-protection frameworks such as HIPAA §164.502(b) and the GDPR data-minimisation principle require that personal data be accessed only to the extent necessary for the immediate purpose. This paper presents the first empirical audit of open-weight LLMs on this access-scope dimension, using the EHRSQL 2024 benchmark over MIMIC-IV. We measure deviations from minimum-necessary access via three static metrics computable without executing queries: Sensitive Column Exposure (SCE), Table Over-Join Rate (TOJ), and Aggregate Identifier Exposure (AIE). Unlike prior work on adversarial attacks, we study default model behaviour under ordinary, benign queries—the over-reach we document requires no attacker and is present from the first interaction. Among twelve open-weight models, a non-trivial fraction fail a basic SQL-generation viability threshold; among nine viable models, SQL-specialised training paradoxically increases over-reach while domain-adapted medical variants achieve the lowest exposure. Schema redaction largely eliminates column-level exposure but increases table-level over-reach for most models. Random in-context demonstrations produce statistically significant over-reach regressions in specific models; BM25-based retrieval resolves most of these failures, though a residual aggregate-identifier exposure risk persists for one model family.