Jun 2026· arXiv.org· Vol abs/2606.29876· 0 citations· 12 references
Computer ScienceBiology
TL;DR
Clinical reasoning graphs, structured graph representations extracted from free-text LLM diagnostic traces using a domain-grounded ontology with 5 node types and 7 edge types show diagnostic competence without schema-scale reasoning consistency, and indicate that final-answer accuracy should be complemented by process-level evaluation.
Abstract
Modern large language models (LLMs) reach 60-70% diagnostic accuracy on complex clinical case benchmarks, but accuracy alone cannot distinguish stable clinically-grounded reasoning from pattern matching. We introduce clinical reasoning graphs, structured graph representations extracted from free-text LLM diagnostic traces using a domain-grounded ontology with 5 node types and 7 edge types. We apply this pipeline to 750 traces from five LLMs across 50 New England Journal of Medicine Clinicopathological Conference cases and three prompt conditions, and test whether diagnostic traces show stable structured reasoning patterns, or diagnostic schemas, for clinically similar cases. We operationalize this as higher graph similarity among clinically similar cases than among clinically dissimilar ones. Across 15 model-condition comparisons, within-cluster and between-cluster composite similarity are nearly equal, and no comparison survives multiple-testing correction; a component-level analysis finds any residual content signal far below schema scale. Graph similarity is also nearly identical for pairs of models that are both correct (0.488) and both incorrect (0.484), suggesting that graph structure captures a dimension not reflected in diagnostic accuracy. Structured reflection prompting increases explicit discriminating-feature analysis within traces (+33%) but does not increase cross-case consistency. These results show diagnostic competence without schema-scale reasoning consistency, and indicate that final-answer accuracy should be complemented by process-level evaluation. We release the ontology, extraction pipeline, validation protocol, and the extracted reasoning graphs and similarity artifacts as resources for structured evaluation of LLM clinical reasoning.
A single LLM verifier lacks sufficient reliability to serve as a stand-alone judge of clinical reasoning at scale, and that structured human oversight remains essential.
Hyunjung Byun, Dahyoun Lee, Munyoung Jung et al.· Journal of medical systems· 1 citation· ⚡1
A clinical knowledge graph construction and refinement framework that combines multi-agent relation proposal, ontology-aware normalization, deterministic evidence scoring, and JEPA-based latent refinement is proposed.
Kushagra Yadav, N. Prabhath, Amit Lamba et al.· 0 citations
ABSTRACT Background Clinical reasoning in contemporary practice often involves ill‐structured, poorly defined problems that span the biopsychosocial domain and require reasoning under high relational complexity. Despite sustained scholarly attention to clinical reasoning, few models are both theoretically grounded and usable for clinicians who face this complexity. At the same time, the World Health Organization Family of International Classifications (WHO‐FIC) offers a rich but complex ontology that was not designed primarily to support clinical reasoning in practice. Aim To develop a two‐stage WHO‐FIC‐based ontological model of clinical reasoning that is conceptually coherent yet remains cognitively tractable in the face of complexity. Methods A conceptual analysis was undertaken. It integrated three strands: (1) WHO‐FIC classification theory, including recent work on harmonisation; (2) empirical and theoretical literature on clinical reasoning; and (3) cognitive theories of bounded rationality, fast‐and‐frugal heuristics, relational complexity, and framing. These strands were used to derive design constraints for a clinically usable ontology. They were then synthesised into a two‐stage, graph‐based model designed to manage the complexity‐coherence trade‐off in clinical reasoning. Results Stage 1 introduces a parsimonious triad—Body, Activity, and Environment—represented as a complete three‐node graph with bidirectional relations. This triad provides an etiologically neutral and cognitively economical starting point for framing complex clinical problems. Stage 2 introduces three emergent constructs—Health Condition, Participation, and Intervention—derived compositionally from Stage 1 relations and aligned with ICD‐11, ICF, and ICHI, respectively. The model supports iterative movement between stages through composition and decomposition. It incrementally increases complexity while keeping relational load within cognitively feasible bounds and preserves the value of diagnosis while reducing its tendency to dominate problem framing. A worked case illustrates how the ontology supports reasoning about complex, time‐dependent problems through repeated movement between stages. Conclusions The proposed ontology addresses key ambiguities within WHO‐FIC, including the relationship between Activity and Participation and the perceived hierarchical privileging of diagnosis. It offers a more coherent and cognitively usable framework while respecting limits on human reasoning and managing the complexity‐coherence trade‐off. The model provides a theoretically grounded heuristic scaffold for clinicians and educators who work with complexity. It also contributes to clinical reasoning literature by emphasising reasoning about concepts, not only cases, with implications for interprofessional education and practice.
David Kellett· Journal of Evaluation In Cli...· 0 citations
Measured extraction performance varied substantially by matching definition, whereas exact-link decisions varied with the availability of matched terminology evidence, support separate evaluation of extraction, retrieval, evidence-grounded linking, and extension candidacy.
Pathology reports anchor cancer diagnosis and staging, yet their narrative structure limits reliable translation into structured, machine-actionable knowledge, creating a bottleneck between expert interpretation and scalable clinical intelligence. Despite decades of clinical natural language processing (NLP) research, pathology text remains among the most complex and consequential sources of medical data to operationalize at scale. Large language models (LLMs) offer new approaches for reading, extracting, and interpreting these reports. We synthesize current LLM work in cancer pathology using a four-level capability framework across the pathology report data lifecycle: (level 1) text preparation and quality checks, (level 2) information extraction, (level 3) guideline-based clinical reasoning, such as TNM staging and registry coding, and (level 4) interpretive synthesis, such as explanations, summarization, or decision support. Rather than grouping studies by NLP task labels, this framework tracks how LLM applications progress from preprocessing and extraction toward higher-level interpretation and synthesis. We followed PRISMA-ScR guidelines and searched four databases through September 2, 2025, identifying 41 eligible studies. Most studies focus on level 2 tasks, with fewer addressing level 3 and level 4 tasks. Encoder-based models, including domain-specific variants such as BioBERT, were commonly used for structured extraction tasks, whereas generative models, including GPT, LLaMA, and Mistral-family models, were increasingly evaluated for prompting-based extraction, staging, and summarization. Reported performance was often high for well-defined extraction tasks, but external validation was uncommon, and metrics varied across studies, limiting direct comparison. Overall, the evidence suggests that success in lower capability levels does not consistently translate to higher-level reasoning, especially when reports are inconsistent, required staging inputs are missing, or clinical assumptions must be inferred, which helps explain gaps between benchmark results and practical adoption. Future work should prioritize robust multi-site validation, clinically meaningful error analysis, transparent evaluation, and privacy-preserving implementation strategies to support safe integration in oncology.
Maryam Seifaddini, Mohammad Beheshti, Steven Richberg et al.· Modern Pathology· 0 citations
Initial experiments on heart-failure-focused clinical question answering show that CGX improves evidence retrieval quality and perceived answer reliability over conventional retrieval methods, while reducing total graph construction time by 69.7% under the same input corpus and hardware setting.
Dat Nguyen, Anh N. Le, Binh T. D. Trinh et al.· Journal of Biomedical Inform...· 0 citations