Evaluating and monitoring reasoning fidelity should be a prerequisite for safe deployment of LLMs in oncology decision support, as endpoint accuracy alone may mask clinically meaningful reasoning failures.
Abstract
Large language models (LLMs) show strong performance on clinical benchmarks, yet their reasoning reliability in real-world oncology care remains unclear. We evaluated LLM reasoning on authentic oncology notes using a novel hierarchical error taxonomy across two retrospective cohorts spanning breast, pancreatic, and prostate cancer. GPT-4 produced reasoning errors in 23.1% of note interpretations, the majority reflecting cognitive bias patterns. Errors were more frequent in recommendation tasks and were strongly associated with guideline-discordant recommendations and lower clinician-rated clinical impact scores. Confirmation bias, anchoring bias, and omission errors were most strongly linked to potentially harmful outputs. Compared to GPT-4, GPT-5.1 demonstrated reduced error rates and improved clinical performance but retained structured reasoning failure patterns. Automated LLM-based evaluators detected error presence but failed to reliably classify subtypes, and a preliminary self-mitigation strategy yielded only modest improvement. Endpoint accuracy alone may mask clinically meaningful reasoning failures. Therefore, evaluating and monitoring reasoning fidelity should be a prerequisite for safe deployment of LLMs in oncology decision support.
Large language models achieve high scores on medical knowledge assessments, yet clinical reasoning requires actively deciding what to investigate under uncertainty. We developed an agentic evaluation framework in hematologic oncology in which models must proactively request clinical data across three sequential rounds before committing to a diagnosis and treatment plan. Across 32 frontier models, the best achieved only 68% overall accuracy. Information utilization, the fraction of available data actually requested, was the strongest predictor of diagnostic accuracy (R = 0.69, P<0.001), yet utilization collapsed from 57% to 26% in the final round, leaving molecular and cytogenetic data critical for treatment selection unexamined. Reasoning traces scored high on a clinical reasoning rubric (91% above threshold) but decorrelated from accuracy, revealing a gap between locally coherent rationales and globally correct conclusions. Error analysis identified search satisficing, anchoring and premature closure as the dominant failure modes, the same cognitive biases that characterize novice clinicians under dual-process models of diagnostic reasoning. These findings demonstrate that the primary limitation of current models in clinical oncology is not insufficient medical knowledge but a systematic failure of information-seeking under uncertainty.
K. Braitsch, L. Schmalbrock, Theresa Weltermann et al.· 1 citation
Clinical decisions, such as diagnosing conditions, prescribing medications, and recommending procedures, are rarely made with absolute certainty. Instead, they reflect probabilistic judgments shaped by evolving patient information and incomplete evidence. However, current EHR systems and knowledge graphs encode such decisions as deterministic triples, lacking a mechanism to represent the subjective confidence inherent in clinical reasoning. We present JudgEHR, a framework for clinical decision confidence estimation that leverages large language models (LLMs) to perform cohort-based collective inference over structured patient records by representing clinical events as knowledge graph triples and integrating them into LLM prompts. JudgEHR groups related clinical concepts into cohorts using LLM-driven relational inference, and then jointly evaluates the plausibility of all clinical decision triples within each cohort by considering patient visit history and background medical knowledge. We apply our method to the MIMIC-III dataset. Our statistical analysis shows that JudgEHR generates semantically consistent confidence scores, with similar concepts receiving closer values, whereas dissimilar replacements yield large confidence differences. Experiments on the MIMIC-III dataset show that incorporating the confidence into a zero-shot LLM-based pipeline improves relative AUROC by ${1 4. 6 \%}$ and AUPRC by 21.8% on the mortality prediction task.
Kexuan Xin, Guillaume Pelat, Jonathan Vitale et al.· International Conference on...· 0 citations
Errors in differential diagnosis often arise while clinicians are generating and comparing candidate explanations. This review examines the use of large language models (LLMs) for this part of diagnostic reasoning. Internal medicine and pediatrics are the main focus; evidence from radiology, surgical subspecialties, infectious disease, and mental health is used to examine how findings change across specialties. Reported performance depends on the clinical setting, the quality of the input, the prompt, model adaptation, and the evaluation design. Some studies place LLMs near trainees and find that they produce wider, better-organized differentials. Experienced clinicians, however, remain more reliable overall. Domain adaptation, external knowledge, and interactive workflows have improved performance in specific evaluations, but hallucinations and automation bias remain, alongside unresolved questions of governance. Current evidence therefore supports clinician-supervised use of artificial intelligence (AI) systems rather than autonomous diagnosis, pending prospective and specialty-specific evaluation.
Yun-Jia Wu, Qi Yan, Dingcheng Tian· AI Medicine· 0 citations
An evaluation framework based on the MedMCQA dataset consisting of two complementary uncertainty settings is proposed, which introduces linguistic uncertainty cues through prompt modifications to simulate ambiguous clinical contexts and observes significant variation across models in their ability to abstain when the correct answer is unavailable.
Maryam Tahermazandarani, Adnan Mahmood, Fahmida Islam et al.· International Conference on...· 0 citations
A multi-dimensional evaluation framework integrating clinical quality and operational efficiency provides more actionable insights than single metric assessments, enabling pragmatic model selection for oncology practice.
M. Halıcı, Serkan Saltürk, Irem Sayin et al.· Scientific Reports· 0 citations
Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation. Error-detection benchmarks are typically constructed by injecting errors into notes, such that each erroneous note has a natural counterpart. Aggregate discriminative metrics (e.g., balanced accuracy or F1) do not exploit this structure. We show that this omission is consequential. In particular, evaluating 15 diverse LLMs on 4 standardized clinical error-detection test sets across 3 languages, we find that 13 of 15 models fall below the level of random pairwise discrimination, even while achieving F1 scores that standard practice would read as moderate. We also observe that the underlying bias patterns differ across languages: the same model can default to"no error"on one language and over-flag errors on another. To diagnose where discrimination breaks down, we further introduce a procedure to score the evidence models cite in their outputs. We find that while models consistently locate error-relevant content, they fail to produce the corresponding correct verdict on the clean counterpart. Finally, we show that F1 and pairwise accuracy are driven in opposite directions by the same underlying bias, so that ranking models by F1 may systematically promote the weakest discriminators. For safety-critical clinical NLP applications, we advocate for supplementing aggregate metrics with paired evaluations in benchmark reporting. Code and analysis scripts are available at https://github.com/healthylaife/paired-clinical-eval.
