A reproducible DSPy+HELM benchmarking study integrates structured prompting into an established evaluation framework and quantifies how prompt choice alone impacts leaderboards.
Asad Aali, Muhammad Ahmed Mohsin, V. Bikia et al.· 1 citation
A scalable framework that automatically generates question--answer pairs from longitudinal EHR notes that supports evaluations that static benchmarks cannot, including the generation of multiple answers that reflect variation in clinician reasoning for robust performance assessment and continuously refreshing benchmark...
J. Cahoon, C. Stanwyck, Sulaiman Somani et al.· arXiv.org· 1 citation
Large language model (LLM)-based clinical assistants are increasingly being integrated into electronic health record (EHR) systems, transforming how clinicians retrieve and synthesize information from patient records. Their safety and utility depend on rigorous evaluation, yet existing benchmarks are manually curated,...
J. Cahoon, C. Stanwyck, Sulaiman Somani et al.· 0 citations
While clinicians knew where and how to annotate complex concepts in patient notes, in twelve of fourteen tasks they faced barriers to replicating those intuitions with LLMs, which highlighted open problems for HCI research to bridge the gaps between AI data work tools and clinical users' needs.
Venkatesh Sivaraman, Rigney E. Turnham, G. Bonano et al.· 0 citations
This work proposes a practical framework for assessing privacy risk in clinical foundation models and illustrates realistic leakage scenarios across deployment settings, map them to legal regimes, and outline complementary technical and legal mitigations.
Sana Tonekaboni, Lena Stempfle, Sasha Ronaghi et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.