It is concluded that while generative AI holds transformative potential to reduce clerical burden and augment clinical reasoning, its successful deployment in emergency medicine requires rigorous attention to clinical safety, health equity, workflow integration, and human‑factors considerations.
The integration of generative artificial intelligence (AI) and large language models (LLMs) into healthcare has accelerated dramatically, with emergency medicine emerging as a particularly dynamic yet challenging domain for clinical deployment. This comprehensive narrative review synthesizes contemporary literature to examine current clinical applications, critical safety considerations, implementation scalability and governance, and future research priorities surrounding generative AI in acute care settings. We examine the current clinical applications, critical safety considerations, implementation scalability and governance, and future research priorities for generative AI in acute care settings. Current emergency medicine applications span automated documentation via ambient clinical intelligence systems, clinical decision support for triage and diagnostic prediction, patient communication tools including discharge summary generation, and multilingual data extraction supporting care transitions. Despite promising efficiency gains - including documented reductions in clinician burnout from 50.6% to 29.4% and modeled documentation time savings of up to 7.1 hours per shift cycle - substantial safety concerns persist, with hallucination rates ranging from 26% to 36% across automated pipelines and systematic misclassification in high-acuity triage tasks. We address automation bias (26% increased risk) and data privacy and governance risks, and identify algorithmic equity as a critical research priority. We conclude that while generative AI holds transformative potential to reduce clerical burden and augment clinical reasoning, its successful deployment in emergency medicine requires rigorous attention to clinical safety, health equity, workflow integration, and human‑factors considerations.
Klaudia Kwolek, Wiktoria Laskowska, M. Pilarek et al.· International Journal of Inn...· 0 citations
Generative AI demonstrably accelerates diagnostic workflows, augments scarce clinical datasets, personalizes communication, and supports discovery pipelines, and the paper concludes with a translational path and research priorities aimed at closing these gaps.
Wael Rahhal· Journal of Data Science and...· 0 citations
Although promising, LLM-based systems are not yet reliable enough for autonomous medical diagnosis, and multiple recommendations for future research are contained to ensure a high level of safety, transparency, and clinical applicability for LLMs and other AI/ML-related technologies and devices.
M. U. K. Gunawardhna, Pirunthavi Wijikumar, D. Weerasinghe· Sri Lankan Journal of Applie...· 0 citations
Large language models and multimodal foundation models are enabling medical artificial intelligence (AI) systems to move beyond isolated prediction and undertake multistep clinical tasks that require planning, tool use, memory, iterative correction, and coordination among specialized agents. However, the scope of agentic AI in medicine remains unsettled, and current evaluation practices are not yet aligned with the requirements of clinical use. We conducted a scoping review with systematic evidence mapping across five electronic sources, screened 1,649 exportable records, and provisionally included 557 unique studies that met predefined criteria for goal-directed task execution, tool use, interaction with external resources, feedback-based refinement, or multi-agent collaboration. The included studies describe single agents that use external tools, workflows supported by retrieval and external knowledge, multimodal agents, and multi-agent systems applied to medical question answering, image interpretation, electronic health record analysis, drug safety, and clinical trial prediction. The evidence base remains dominated by public benchmarks, simulated settings, retrospective datasets, and small-scale expert evaluation. Process reliability, evidence traceability, uncertainty, safety, workflow impact, and external validity are evaluated less consistently. Clinical translation will depend on clearer definitions, reproducible evaluation, auditable oversight, interoperable system design, and prospective validation in real-world clinical workflows.
Zheng Tong, Yang Liu, Wanshu Fan et al.· 0 citations
Generative artificial intelligence (GenAI), particularly large language models, is rapidly emerging as a transformative tool in medicine, including nephrology, with applications spanning clinical decision support, documentation, patient education, and research. Unlike traditional machine learning systems, GenAI produces probabilistic, context-dependent outputs, introducing novel opportunities but also distinct risks.
This review examines key risk domains associated with the integration of GenAI into kidney care. These include data-related challenges such as bias, limited representativeness, and data quality issues; technical limitations such as hallucinations and prompt-dependent variability; and cognitive risks, including automation bias and overreliance. The differences between human clinical reasoning and AI-generated outputs further complicate safe implementation, particularly in high-stakes decision-making contexts. In addition, broader systemic concerns are addressed, including inequities in access, linguistic and infrastructural disparities, and the environmental footprint associated with large-scale AI systems.
We highlight that these risks are not isolated but interconnected, with potential implications for patient safety, clinical decision-making, and health system equity. While GenAI may enhance efficiency and support clinical workflows, its outputs require critical appraisal and validation within the clinical context.
Safe and responsible implementation will depend on a combination of technical safeguards, structured prompting, rigorous validation, and human-in-the-loop oversight. Regulatory frameworks provide an essential foundation, but clinical accountability remains central.
Ultimately, the integration of GenAI in nephrology should prioritize safety, equity, and sustainability, ensuring that technological innovation translates into meaningful and responsible improvements in patient care.
Elizabeth R Viera Ramírez, Leonor Fayos de Arizón, Roser Torra et al.· Clinical Kidney Journal· 0 citations
Recent advances in Large Language Models (LLMs), driven by transformer architectures such as Generative Pre-Trained Transformer (GPT), are opening new frontiers in healthcare Artificial Intelligence (AI) by enabling clinically relevant interactions between patients and clinicians. Yet persistent challenges—including limited real-time knowledge access, safety concerns and insufficient patient-centered contextualization—indicate that current systems often fall short in delivering efficient and reliably personalized responses (Li et al., 2025; Cascella et al., 2023). To address these gaps, Retrieval-Augmented Generation (RAG) can couple LLMs with external knowledge sources at inference time, producing responses that are more grounded, up-to-date, and tailored to individual patient needs. This systematic review examines how personalization is operationalized in LLM-only and RAG-enhanced healthcare AI systems. We searched multiple scholarly sources and included 20 studies in the final qualitative synthesis. We compare personalization strategies, evaluation practices, and trustworthiness challenges, with emphasis on healthcare-specific issues such as privacy, safety, and clinically meaningful context integration. We analyze how current evaluation frameworks assess these systems and identify limitations in their ability to reflect clinically meaningful performance. We further include a brief MedQuAD-based illustrative case study to highlight limitations of current evaluation metrics, demonstrating that strong benchmark performance does not necessarily correspond to clinically reliable or deployment-ready behavior. The review finds that personalization remains inconsistently defined and weakly evaluated in healthcare AI, with most systems implementing retrieval-grounded adaptation rather than true patient-specific personalization. Furthermore, relatively few studies directly evaluate hallucination, patient safety, clinician validation, or deployment robustness, revealing a critical gap between commonly reported performance metrics and the requirements of clinically reliable, safe, and real-world deployable decision-support systems.
Manal Althobaiti, Minhee Jun· Frontiers in Artificial Inte...· 0 citations