Skip to content
Book Open access

Agents in the Wild: Where Research Meets Deployment

Jul 2026 · Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · 0 citations · 62 references
Computer Science

TL;DR

Through applied case studies in pharmaceutical discovery and financial systems, common design patterns that make agentic systems successful are analyzed, and practical mitigation strategies for failure modes are discussed, such as verification pipelines, fallback mechanisms, and human-in-the-loop supervision.

Abstract

Agentic systems — large language model (LLM)-based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents — are rapidly transitioning from research prototypes to production-scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi-agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as verification pipelines, fallback mechanisms, and human-in-the-loop supervision. Attendees will gain a comprehensive view of the field along with concrete design patterns, evaluation checklists, and templates for safe and reliable deployment across industries.

Read PDF

Similar papers

Review Open access Aug 2026

From Language Models to Agentic AI: A Survey of Autonomous, Action-Enabled, and Collaborative LLM Agents

A unified, taxonomy-driven, and deployment-oriented survey of agentic AI systems, synthesizing recent advances through a modular reference architecture and a four-dimensional taxonomy that characterizes agents along the axes of autonomy, tool use, collaboration, and safety–governance is presented.

Sparsh Bajoria, Shreyanshu Ranjan, Adhitya M et al. · 0 citations
Preprint Aug 2026

Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback

Agent Gym is introduced, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop and introduces the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency.

Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge et al. · 0 citations
Preprint Aug 2026

Towards Risk-free AI Agent Deployment

It is argued that risk-free deployment must be grounded in the agent's trajectory: the recorded sequence of reasoning steps, tool invocations, and environmental observations, and the absence of adequacy metrics.

Yintong Huo, Rangeet Pan, Abhik Roychoudhury · 0 citations
Preprint Jul 2026

Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents

This paper synthesizes 27 benchmark, taxonomy, and audit papers (2023-2026), spanning 19 distinct benchmarks, into a cross-cutting taxonomy of agent limitations, the first synthesis that integrates evidence across tool use, planning, long-horizon reasoning, multi-agent coordination, safety, and measurement validity into a single, unified taxonomy of LLM agent limitations.

Wael S. Albayaydh, Rui Zhao, Ivan Flechais · 1 citation
Conference Open access Jul 2026

AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments

AgentGym2 is presented, a new evaluation framework with task instances grounded in real-world end-to-end working demands that measures agents'ability to execute end-to-end procedures, discover tools via exploration, compose tools for unseen tasks, and remain robust to noisy and underspecified information.

Zhiheng Xi, Dingwen Yang, Jiaqi Liu et al. · 1 citation
Book Open access Aug 2026

Enterprise AI Agents: From Prototypes to Production

This workshop aims to bring together researchers and practitioners to examine how enterprise AI agents can successfully move from prototypes to production, and focuses on three pillars: 1) Agent architectures and systems; 2) Enterprise applications and deployments; 3) Evaluation and governance.

Min Du, Anbang Xu, Jasmine Jaksic et al. · 0 citations