Agentic AI and Multi-Agent Collaboration in Healthcare: A Comprehensive Survey of Architectures, Clinical Safety, and Future Directions

Abstract

In the healthcare and clinical domain, artificial intelligence (AI) is evolving from earlier models that primarily predicted outcomes or generated content toward agentic AI systems that demonstrate the capability to make decisions and complete tasks autonomously. Previous research on AI has contributed significantly to disease identification, deep learning applications, large language models (LLMs), and generative AI. These systems primarily function as assistive tools, as they generate text or predictions without directly interacting with clinical infrastructures. Therefore, recent research trends are increasingly oriented toward agentic AI systems that extend beyond traditional predictive and generative model performance. This manuscript provides a detailed review of the current state of agentic AI, starting with the evolution of AI and the concept of a medical agent. A medical agent refers to an intelligent AI system designed to assist in clinical or administrative tasks by analyzing data, supporting decision making, and interacting with healthcare environments. Its underlying agentic AI architecture integrates planning, memory, reasoning, and environmental interaction to enable autonomous tool use, multi-agent collaboration, and continuous perception decision action loops across diverse healthcare applications and clinical workflows. The review further examines safety mechanisms, including human-in-the-loop oversight, self-verification strategies, and regulatory alignment frameworks, which are designed to ensure reliability, accountability, compliance, and safe deployment in regulated healthcare environments. Our findings indicate that a large number of AI agents have been introduced in various manuscripts for healthcare applications; however, fully autonomous systems remain challenging to achieve, as AI still faces several limitations related to reliability, interpretability, data dependency, and integration within complex clinical workflows. In response to these challenges, this survey shifts the focus from task-specific model performance to system-level autonomy and workflow orchestration, providing a structured foundation for understanding the design, deployment, governance, and limitations of agentic AI systems in modern healthcare ecosystems.