Aug 2026· Frontiers in Public Health· Vol 14· 0 citations· 37 references
Medicine
TL;DR
These findings identify unsupported reassurance as measurable evidence-boundary behavior in LLM drug-risk assessment and establish a reproducible framework for auditing adherence to an externally supplied evidence boundary, defined by PubMed evidence classification and enforced by prompt policy rather than inferred independently by the model.
Abstract
Introduction Large language models (LLMs) are entering drug-safety and regulatory workflows, yet their behavior at the boundary between unavailable evidence and risk reassurance remains poorly characterized. Drug-induced liver injury (DILI) is a stringent setting because evidence is fragmented across labels, case reports, mechanistic studies, and curated knowledge bases, while unsupported low-risk reassurance can be consequential. Methods We evaluated five LLMs on a fixed 47-drug DILI risk-assessment panel using 1,410 parsed responses from closed-book answering and an evidence-gated protocol that restricted responses to supplied PubMed-derived evidence. Curated DILI resources were excluded from prompts and used only for evaluation. After filtering, 38 drugs had no direct DILI decision-support evidence in the supplied packet, and 9 had direct DILI-relevant evidence; a post hoc PubMed title/abstract recall stress test identified five additional drugs with recoverable direct DILI evidence outside the packet. Results In the no-direct-evidence slice defined by the supplied packet, closed-book models rarely abstained, with drug-level abstention ranging from 5.3% to 33.3%; the evidence-gated protocol required abstention, which all models followed for every no-direct-evidence drug. The same pattern held for recent or low-recognition drugs, where evidence-gated abstention reached 92.0% to 100.0% vs. 8.0% to 49.3% under closed-book answering. Closed-book models also produced high-confidence low-risk responses for DILI-positive drugs, a label-discordant pattern largely removed by evidence gating. Independent expert review of selected responses showed that label discordance did not always imply a clinically unreasonable low-risk category, but identified unsafe reassurance through overconfident wording and under-cautious responses in selected cases. When direct DILI evidence was provided, all models preserved citation-grounded non-abstaining answers. However, they differed in how often they committed to a conclusive rather than an uncertain risk category. Citation-bearing evidence-gated responses cited only the supplied PubMed identifiers and achieved 91.2% to 100.0% concordance with the supplied grade. Discussion These findings identify unsupported reassurance as measurable evidence-boundary behavior in LLM drug-risk assessment and establish a reproducible framework for auditing adherence to an externally supplied evidence boundary, defined by PubMed evidence classification and enforced by prompt policy rather than inferred independently by the model.
Background/Objectives: Pharmacovigilance workflows rely heavily on unstructured text across diverse sources. Here, we systematically reviewed how large language models (LLMs) are being explored as support tools for adverse drug reaction (ADR) detection, extraction, triage, and documentation, highlighting their potential for precision medicine and big data-enabled safety monitoring. Methods: Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 guidelines, we systematically searched PubMed, Scopus, and Web of Science for studies published between January 2022 and March 2026. Ultimately, 83 empirical studies satisfied the inclusion criteria. A narrative synthesis was conducted to address methodological heterogeneity across these studies. Results: LLM applications were concentrated in constrained information-extraction and classification tasks, including signal evaluation, clinical-note extraction, social media surveillance, and literature screening. Quantitative performance varied substantially by system design: error-correction prompting yielded an F1-score of 0.921 for ADR named entity recognition, whereas retrieval-augmented generation improved data-retrieval accuracy from 8.3% to 78.3%. Most studies were retrospective, benchmark-based, or proof-of-concept evaluations. Across 581 paired pre-consensus domain judgements, observed inter-rater agreement was 90.4% and Cohen’s κ was 0.837 (95% CI 0.772–0.895). Hallucination, low specificity, prompt sensitivity, narrow datasets, and weak external validation remained common limitations. Conclusions: Current evidence supports supervised, task-specific applications of LLMs for extraction, triage, retrieval, and documentation rather than autonomous pharmacovigilance decision-making. Prospective evaluation, external validation, transparent reporting, and accountable human oversight are required before high-stakes clinical or regulatory deployment.
Tae You Kim, Won-Sik Oh, Dong-Hwa Jeong· Diagnostics· 0 citations
Clinical feature extraction from pathology reports is challenging because relevant evidence may be distributed across coded and narrative fields and depend on specimen attribution, negation, ancillary findings, and diagnostic context. We retrospectively evaluated the NimbleMind Multi-Agent System (nMAS), a configurable workflow that separates clinician-defined field specifications from extraction models and returns report-level predictions with source-linked evidence. The study included 54 dummy gastric biopsy pathology reports from Singapore and four binary target fields, yielding 216 feature-case decisions. nMAS correctly classified 213 of 216 decisions (98.61\%), and all evidence spans associated with correct predictions occurred verbatim in the corresponding source reports. All three errors occurred in the two context-dependent \textit{H. pylori}-related fields requiring negation handling or diagnostic attribution. A single-model UMA-style comparator produced the similar label-level performance and error pattern. These findings do not demonstrate predictive superiority for the multi-agent architecture.Rather, the contribution of nMAS lies in workflow integration and traceability through configurable field specifications, complexity-based routing, report-level aggregation, and source-text validation within a clinician-reviewable workflow. Larger multi-institutional studies should assess generalizability, semantic evidence quality, adaptation effort, and clinician verification time.
Yufan Wang, Anit Kumar Sahu, Y. Ng et al.· 1 citation
BACKGROUND
Accurate detection of drug-drug interactions (DDIs) is a fundamental component of safe medication management. Traditional rule-based clinical decision support systems for DDI identification lack higher-order reasoning and contribute to alert fatigue. Large language models (LLMs) have potential for DDI identification but may hallucinate, inconsistently identify interactions, and provide overly confident responses despite uncertainty. Prior studies have emphasized accuracy, but few have examined whether LLM uncertainty expression aligns with error risk.
OBJECTIVE
To evaluate LLM performance in DDI identification using a clinician-validated dataset and to assess whether prompt-based mitigation strategies improve knowledge-aware uncertainty expression, defined as alignment between refusal behavior and likelihood of error.
METHODS
We developed a clinician-curated DDI identification task consisting of 250 medication lists, each containing 1 clinically relevant interacting drug pair, to evaluate 5 LLMs: GPT-5-Chat, GPT-4o-mini, Gemma-27B, LLaMA3-70B, and Qwen3-32B. Models were evaluated using 3 prompt formats and a zero-shot approach, with no task-specific training or examples provided. Prompts were designed to encourage uncertainty acknowledgment, including a patient safety-focused mitigation prompt to support clinically appropriate and cautious responses. Each case was run 9 times per prompt condition. The primary outcome was the Refusal Index (RI), which quantifies alignment between model refusal behavior and likelihood of error. Secondary outcomes included overall accuracy, accuracy given attempted, refusal rate, self-consistency, F score, weighted score, and entropy.
RESULTS
Across models, overall DDI identification accuracy ranged from 54.1% to 83.7%. GPT-5-Chat demonstrated the highest overall accuracy and self-consistency, whereas Quen3-32B demonstrated the lowest accuracy but the highest refusal rates. Alignment between refusal behavior and likelihood of error was weak to moderate (RI range 0.104-0.574) and varied by model. Prompt-based mitigation strategies produced inconsistent effects on RI and did not reliably recalibrate uncertainty behavior. Notably, higher overall accuracy and response stability did not consistently correspond to stronger knowledge-aware uncertainty expression. Qwen3-32B and GPT-4o-mini increased refusal rates under mitigation prompting but not in situations where responses were more likely to be incorrect.
CONCLUSIONS
Substantial variability exists in both DDI identification performance and uncertainty calibration across LLMs. Prompt design alone was insufficient to consistently improve knowledge-aware uncertainty expression. Because safe deployment of LLMs in medication management depends not only on accuracy but also on appropriate deferral when error risk is elevated, multidimensional evaluation frameworks are essential before clinical use.
A. Tilley, Brian Murray, K. Henry et al.· Journal of Managed Care & Sp...· 0 citations
It is concluded that LLM-based decision-support tools hold substantial promise as complementary — rather than autonomous — decision-support systems capable of transforming medication safety and pharmacy practice.
K. K. Kumar, Koyya Gowtham Reddy, K. Reddy· International Scientific Jou...· 0 citations
Abstract Background Large language models (LLMs) can accurately classify biomedical documents, but strong benchmark performance does not establish that predictions are grounded in the supplied text. In biomedical literature tasks, titles, abstracts, digital object identifiers (DOIs), journal metadata, and trial identifiers may have been seen during pretraining and can trigger parametric knowledge or learned associations. Objective This study aimed to test whether oncology randomized trial success classification is driven by abstract evidence or by identifier-conditioned prior knowledge, and assess whether models follow counterfactual outcome evidence when it conflicts with original trial identifiers. Methods We evaluated 250 two-arm oncology randomized controlled trials from 7 major journals published between 2005 and 2023, each with a single primary endpoint and previously adjudicated positive or negative ground-truth label. The corpus included 58.4% (146/250) positive and 41.6% (104/250) negative trials. GPT-5.2, Gemini 3 Flash, and Claude Opus 4.5 were queried via vendor APIs under default settings using a single-token output instruction. For each trial, we created 5 deterministic input conditions: title+abstract, title only, DOI only, counterfactual title+abstract in which the primary endpoint outcome statement was minimally flipped, and the same counterfactual input paired with the original DOI to create an identifier-text conflict. Performance was assessed using valid format rate, accuracy, sensitivity, specificity, and F1-score. Results The models showed high format adherence, with valid prediction rates of 97.2% to 100%. In the title+abstract condition, all models achieved high and balanced performance (accuracy and F1-score=0.96-0.97; sensitivity=0.96-0.97; specificity=0.96-0.98). Removing evidence reduced performance stepwise: title-only accuracy and F1-score fell to 0.79 to 0.88, and DOI-only performance fell to 0.63-0.67, exceeding the 58.4% majority class baseline but indicating limited identifier-driven signal. Counterfactual edits were concentrated in outcome-bearing text, with the Results and Conclusions sections modified for all trials, whereas the titles and Methods sections required edits in only 5.2% (13/250) and 1.6% (4/250) of trials. Against inverted labels, models followed counterfactual evidence with near-ceiling performance (accuracy and F1-score=0.96-0.99). Reintroducing the original DOI caused little change for GPT-5.2 (accuracy and F1-score=0.99) but modestly reduced F1-scores for Gemini (0.97) and Claude (0.95), mainly through lower sensitivity. Conclusions The evaluated LLMs robustly followed explicit end point statements in abstracts, including when those statements contradicted original trial outcomes. However, above-chance title-only and DOI-only performance, together with small decrements under counterfactual DOI conflicts, showed that identifiers can carry predictive signal and occasionally compete with textual evidence. Progressive content removal combined with counterfactual identifier-text conflicts offers a practical, reproducible audit for grounding in biomedical LLM evaluations.
P. Windisch, C. Koechli, Fabio Dennstädt et al.· JMIR AI· 0 citations
Findings show that clinical LLM explainability has shifted toward fluent generative rationales, but evidence that such explanations reflect model reasoning remains limited, and three regulatory priorities are highlighted: prioritizing explanations that enable independent verification or logic auditing over plausibility-only rationales; preferring inspectable models where regulatory documentation is required; and prospectively validating explanations in clinical workflows before scaling.