Aug 2026· PLOS Digital Health· Vol 5, pp. e0001621· 0 citations· 72 references
Medicine
TL;DR
This narrative review outlines the limitations of traditional evaluation frameworks and proposes a paradigm shift toward adaptive, iterative, and context-specific assessment methodologies that can progress from a promising technology to reliable clinical tools that improve patient outcomes, support clinical decision-making, and uphold ethical standards in routine practice.
Abstract
Artificial intelligence (AI) predictive models demonstrate potential for transforming clinical decision-making across medicine. However, conventional randomized controlled trials (RCTs), the gold standard for evaluating medical interventions, are ill-suited for clinical AI tools due to their static design, lengthy timelines, and inability to accommodate algorithms that evolve and adapt to changing clinical contexts. In this narrative review, we outline the limitations of traditional evaluation frameworks and propose a paradigm shift toward adaptive, iterative, and context-specific assessment methodologies. Evaluating clinical AI in practice requires three interdependent but epistemologically distinct activities: performance monitoring, which tracks the technical characteristics of the deployed model (calibration, discrimination, data drift, alert burden, fairness, workflow fidelity); clinical impact monitoring, which observationally and prospectively tracks whether the initial clinical benefit appears sustained over time; and scientific evidence generation, which produces causal estimates of deployment effects on patient outcomes through pragmatic, adaptive trial designs and causal inference techniques. We propose a predictive-AI-specific framework that links performance monitoring, clinical impact monitoring, evidence generation, causal estimands, and governance of model updates into one coherent decision pathway for clinicians and trialists. We present a governance-driven escalation protocol specifying when monitoring signals should trigger formal evidence generation, a decision pathway mapping signal types (performance or clinical impact) to trial design classifications, and a guide to causal inference methods for clinical AI trials. Drawing from adaptive platform and pragmatic trial designs, we recommend continuous monitoring approaches that prioritize patient-centered outcomes, health equity, and workflow integration over narrow performance metrics, and provide actionable steps to design a clinical AI trial. Successful implementation requires clinician engagement, transparency, and ongoing education regarding AI capabilities and limitations. Within this new evaluation paradigm, predictive AI can progress from a promising technology to reliable clinical tools that improve patient outcomes, support clinical decision-making, and uphold ethical standards in routine practice.
Background: Artificial Intelligence (AI) has emerged as one of the most transformative technologies in modern healthcare, significantly influencing clinical decision-making processes. AI-driven systems, including machine learning algorithms, deep learning models, natural language processing, and clinical decision support systems, have demonstrated the ability to analyze vast amounts of healthcare data, identify patterns, predict outcomes, and support evidence-based clinical decisions. As healthcare systems face increasing patient complexity, workforce shortages, and demands for improved quality of care, AI offers opportunities to enhance diagnostic accuracy, treatment planning, risk prediction, and healthcare efficiency. However, concerns regarding algorithm transparency, ethical accountability, data privacy, bias, and professional acceptance continue to challenge its widespread adoption.
Objectives
To systematically review the evidence regarding the role of artificial intelligence in clinical decision-making across healthcare settings.
To identify the benefits and effectiveness of AI-supported clinical decision-making systems in improving patient outcomes.
To examine challenges, ethical considerations, and barriers associated with AI implementation in clinical decision support.
To evaluate implications for healthcare professionals, particularly nurses and physicians, in AI-assisted clinical practice.
Methods: This systematic review was conducted according to Joanna Briggs Institute (JBI) methodology and reported following PRISMA 2020 guidelines. Electronic databases including PubMed, Scopus, Web of Science, CINAHL, ScienceDirect, and Google Scholar were searched for peer-reviewed studies published between 2019 and 2025. The review question was structured using the PICOT framework. Methodological quality was assessed using appropriate JBI Critical Appraisal Tools. Due to heterogeneity among study designs and outcomes, findings were synthesized narratively.
Results: Thirty-five studies met the inclusion criteria. Evidence demonstrated that AI-assisted clinical decision-making significantly improved diagnostic accuracy, risk stratification, treatment planning, predictive analytics, medication management, and workflow efficiency. AI systems were particularly effective in radiology, oncology, cardiology, intensive care, and chronic disease management. However, challenges related to explainability, algorithmic bias, legal liability, data privacy, workforce adaptation, and ethical governance were consistently reported.
Conclusion: Artificial intelligence has substantial potential to strengthen clinical decision-making and improve healthcare outcomes. Successful integration requires robust regulatory frameworks, transparent algorithms, ethical governance, clinician training, and preservation of patient-centered care principles. AI should function as a supportive tool that enhances rather than replaces human clinical judgment.
Arun James, Dr. Jomon Thomas, Dr. Deepika Verma et al.· PAIN, JOINTS, SPINE· 0 citations
Introduction Artificial intelligence (AI) is reshaping healthcare, enabled by advances in computing, affordable data storage, and the widespread adoption of electronic health records (EHRs). Machine learning (ML), deep learning (DL), and natural language processing (NLP) are increasingly used for disease diagnosis, risk prediction, and treatment planning. Objective This systematic review aimed to examine AI applications across clinical domains from 2020 to 2025, assess their diagnostic accuracy and clinical performance relative to standard practice, identify key implementation barriers including regulatory compliance, algorithmic fairness, and transparency challenges, and compare validation practices and methodological quality with earlier systematic reviews. Methods This systematic review followed PRISMA 2020 guidelines. We searched five databases (PubMed, IEEE Xplore, Web of Science, Springer, and Semantic Scholar) for studies published from January 2020 to September 2025. We included original clinical AI studies that reported prospective validation and/or external validation. Results Twenty studies met the inclusion criteria. Publication volume peaked in 2024 (n = 7, 35.0%). DL approaches were most common (n = 12, 60.0%), with convolutional neural networks (CNNs) frequently applied to medical imaging tasks. By clinical domain, 30.0% of studies focused on radiology (n = 6), 20.0% on oncology (n = 4), and 15.0% on cardiology (n = 3). For imaging-based diagnostic models, the descriptive median performance across individual studies was 0.91 AUC (no formal meta-analysis was conducted due to heterogeneity in study designs, populations, and outcome metrics). The most frequently reported challenges were regulatory compliance (55.0%, n = 11), limited algorithmic transparency (40.0%, n = 8), data quality limitations (35.0%, n = 7), and barriers to clinical integration (30.0%, n = 6). Conclusions AI demonstrates strong potential to improve the effectiveness, safety, and quality of healthcare. However, broader clinical adoption remains constrained by regulatory requirements, interpretability gaps, data quality issues, and workflow integration challenges, underscoring the need for stronger validation practices and more implementation-focused research.
Ghulam Hussain Noori, Shaista Bibi, Seung Won Lee· Inquiry : a journal of medic...· 0 citations
Artificial intelligence has been proposed as a corrective to three persistent problems in mental healthcare: diagnostic imprecision, the trial-and-error character of treatment selection, and the episodic nature of clinical monitoring. The volume of primary research has expanded rapidly, yet few tools have altered routine practice. This critical narrative review examines evidence across the three domains in which artificial intelligence has been most extensively applied to mental health, namely diagnostic classification and risk detection, treatment personalisation, and continuous patient monitoring, and asks why demonstrated technical performance has so rarely converted into demonstrated clinical benefit. Literature was identified through a bibliographic metadata registry, a biomedical citation index, targeted searching of scholarly and institutional sources, and backward and forward citation tracking, covering January 2015 to 11 June 2026, with earlier work retained where conceptually necessary. Evidence was appraised for design adequacy, validation strategy, sample representativeness, outcome definition and reporting transparency, then synthesised thematically rather than study by study. Three findings recur. Apparent accuracy is systematically inflated by internal validation, small and selected samples, and reference standards of limited reliability; where external validation has been attempted, discrimination frequently falls towards chance. The three domains differ markedly in evidential maturity, since monitoring and conversational intervention now rest on randomised evidence and pooled effect estimates, whereas diagnostic classification and treatment-response prediction remain largely at the model-development stage. The binding constraints on translation are infrastructural and epistemic rather than algorithmic, encompassing narrow training populations, unreliable outcome labels, absent prospective evaluation and immature governance. Unresolved questions include whether any model confers benefit over routine care in prospective use, how algorithmic outputs should enter clinical judgement, and how safety should be established for generative systems operating outside professional supervision. Progress will depend less on model refinement than on representative longitudinal datasets, standardised outcome definitions, prospective impact evaluation and governance capable of distinguishing wellness products from clinical instruments.
Oyebode Mary Oluwabunmi, Anyebe Daniel Ameh, Jacob Miracle Godswill et al.· Journal of medicine and heal...· 0 citations
Large language models and multimodal foundation models are enabling medical artificial intelligence (AI) systems to move beyond isolated prediction and undertake multistep clinical tasks that require planning, tool use, memory, iterative correction, and coordination among specialized agents. However, the scope of agentic AI in medicine remains unsettled, and current evaluation practices are not yet aligned with the requirements of clinical use. We conducted a scoping review with systematic evidence mapping across five electronic sources, screened 1,649 exportable records, and provisionally included 557 unique studies that met predefined criteria for goal-directed task execution, tool use, interaction with external resources, feedback-based refinement, or multi-agent collaboration. The included studies describe single agents that use external tools, workflows supported by retrieval and external knowledge, multimodal agents, and multi-agent systems applied to medical question answering, image interpretation, electronic health record analysis, drug safety, and clinical trial prediction. The evidence base remains dominated by public benchmarks, simulated settings, retrospective datasets, and small-scale expert evaluation. Process reliability, evidence traceability, uncertainty, safety, workflow impact, and external validity are evaluated less consistently. Clinical translation will depend on clearer definitions, reproducible evaluation, auditable oversight, interoperable system design, and prospective validation in real-world clinical workflows.
Zheng Tong, Yang Liu, Wanshu Fan et al.· 0 citations