Large language models are increasingly deployed as natural-language interfaces to clinical databases, yet their evaluation has focused almost exclusively on query correctness rather than access minimality. Data-protection frameworks such as HIPAA §164.502(b) and the GDPR data-minimisation principle require that personal data be accessed only to the extent necessary for the immediate purpose. This paper presents the first empirical audit of open-weight LLMs on this access-scope dimension, using the EHRSQL 2024 benchmark over MIMIC-IV. We measure deviations from minimum-necessary access via three static metrics computable without executing queries: Sensitive Column Exposure (SCE), Table Over-Join Rate (TOJ), and Aggregate Identifier Exposure (AIE). Unlike prior work on adversarial attacks, we study default model behaviour under ordinary, benign queries—the over-reach we document requires no attacker and is present from the first interaction. Among twelve open-weight models, a non-trivial fraction fail a basic SQL-generation viability threshold; among nine viable models, SQL-specialised training paradoxically increases over-reach while domain-adapted medical variants achieve the lowest exposure. Schema redaction largely eliminates column-level exposure but increases table-level over-reach for most models. Random in-context demonstrations produce statistically significant over-reach regressions in specific models; BM25-based retrieval resolves most of these failures, though a residual aggregate-identifier exposure risk persists for one model family.
Alberto Ballesteros-Rodríguez, Lino González-García, Miguel-Ángel Sicilia et al.· Electronics· 0 citations
Shapelet-based classifiers offer structural interpretability: discriminative subsequences form an inspectable vibration-pattern vocabulary and a shallow decision-tree ensemble produces traceable fault-type predictions. We impose explicit interpretability constraints on the model and apply Bayesian optimisation within this bounded region; the primary experimental question is whether these constraints carry a performance penalty relative to an unconstrained baseline. Evaluation uses recording-level cross-validation on CWRU (Case Western Reserve University) and MFPT (Machinery Failure Prevention Technology) bearing datasets with Hilbert envelope demodulation. The central finding is that the constraints impose no systematic performance penalty: the shapelet classifier matches ROCKET, a non-interpretable baseline, on both datasets, with cross-validated mean F1 differences smaller than the fold-to-fold standard deviation. To further characterise the selected models under the controlled laboratory conditions studied here, we assess probability calibration and conformal prediction coverage as secondary analyses. Raw probability estimates are well-calibrated, but Platt scaling degrades under cross-severity distribution shift; split conformal prediction yields valid coverage on CWRU but fails on MFPT due to class-proportion mismatch across recording-level splits. Together, these results show that structural constraints supporting interpretability are compatible with competitive performance, and identify the conditions under which reliability tools succeed and fail in this setting.
Lino González-García, Luis Usero, Miguel-Ángel Sicilia et al.· Electronics· 0 citations
