2026· Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track)· pp. 436-446· 1 citation· 39 references
TL;DR
Use across 100 Account Managers validates that LLM-based agents achieve enterprise-grade reliability when augmented with robust web automation, comprehensive document understanding, and systematic SOP refinement.
Abstract
While Large Language Models excel at reasoning and language understanding, they struggle with multi-step operational workflows requiring precise procedural adherence, which is fundamental for industrial automation. Existing SOP-guided agents assume well-defined procedures and structured APIs, failing to address enterprise realities like incomplete SOPs, dynamic web interfaces, and unpredictable document formats. We present Agent-Ops, an end-to-end multi-agent framework automating Standard Operating Procedures in e-commerce. Agent-Ops contributes: (1) SOP Groomer , a human-AI framework transforming ambiguous documentation into automation-ready specifications, (2) WebAgent , achieving 91.3% task completion and 86.5% execution consistency through demonstration-based learning, and (3) a Document Verification Agent performing multi-lingual validation across tax in-voices, certificates, and supply chain documents with 94.2% accuracy. Deployed across seven SOP categories in three geographic regions, Agent-Ops achieves 85-97% end-to-end accuracy while reducing case resolution time by 83% reduction. Use across 100 Account Managers validates that LLM-based agents achieve enterprise-grade reliability when augmented with robust web automation, comprehensive document understanding, and systematic SOP refinement.
Enterprise Resource Planning (ERP) systems record transactions reliably but still delegate almost all operational decision-making to human specialists, because classical rule-based automation cannot reason about exceptions and monolithic AI assistants degrade when asked to coordinate across functional boundaries. This paper presents Agentic ERP, an expert-system architecture that combines role-aligned large-language-model (LLM) agents with a risk-tiered human-in-the-loop harness and a graph-based orchestrator to execute end-to-end business workflows on a production ERP backend. First, autonomous ERP operation is formulated as a constrained sequential-decision problem over a structured enterprise state, with a decomposition argument linking role-aligned agents to a measurable reduction in per-step tool-selection complexity. Second, a graph-based Planner--Executor--Reflector--Responder orchestration decouples generation from evaluation through externalised grading criteria and sprint contracts, packaging recent harness-engineering principles as inspectable expert-system artefacts. Third, the system is evaluated at three levels: a scenario-based task suite, a comprehensive comparison of six orchestration paradigms on cross-functional crisis tasks, and a 365-day agent-in-the-loop simulation against rule-based RPA and no-intervention baselines. Across these levels the proposed multi-agent method is significantly better than the baseline, and the system sustains a simulated year of operation with zero stockouts while the rule-based baseline accumulates hundreds under the same demand stream. The work shows that role-aligned LLM agents under human oversight can move an ERP system from passively recording transactions to actively executing operational decisions, and it provides a reference architecture and an evaluation protocol for autonomous enterprise resource planning.
Zhihao Liu, Tianyu Wang, X. Wang et al.· 0 citations
UMA, a Unified Multi-Agent Framework for enterprise AI systems, is introduced, designed to support the complete lifecycle of agentic systems, including deployment, orchestration, execution, monitoring, and return-on-investment (ROI) realization.
Umamaheswara Rao Kukkala· International Journal of Inn...· 0 citations
Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coordinated use of tools such as notebooks, IDEs, terminals, browsers, and databases within real operating environments. Yet existing benchmarks lack real-computer interaction and do not evaluate whether agents can execute complete end-to-end data-science workflows in realistic computing environments, failing to capture the multi-stage, multi-tool nature of data-science practice. We introduce DSAgentBench, the first benchmark to evaluate whether agents can automate full data-science workflows inside real computer environments. DSAgentBench contains 275 diverse tasks covering the entire data-science life-cycle, reflecting the complexity and tool coordination required in practice. Each task requires grounding decisions in intermediate outputs and coordinated tool use, and includes a deterministic evaluator that verifies analytical correctness, visual outputs, and model performance rather than code-only execution. Our extensive experiments with 15 closed- and open-source models show that even the strongest agent, Claude-4.6-Sonnet, achieves only 56.70% task success, while all open-source agents remain below 1%, frequently failing at tool orchestration, OS grounding, and multi-step reasoning. These results reveal a substantial capability gap between current agentic systems and real data-science workflows, positioning DSAgentBench as a foundation for developing grounded, verifiable, autonomous data-science agents. We release DSAgentBench at https://github.com/vis-nlp/DSAgentBench.
Mizanur Rahman, Mohammed Saidul Islam, Ridwan Mahbub et al.· 0 citations
Large Language Models (LLMs) have spurred the development of agentic artificial intelligence systems that can reason autonomously, plan, use tools, integrate memory, and carry out multi-step tasks. Unlike conventional prompt-response systems, LLM-based agents extend generative models to goal-oriented architectures that can decompose complex objectives, interact with external environments, and coordinate actions in iterative workflows. The survey systematically reviews LLM-based agent frameworks, multi-agent collaboration architectures, internal communication mechanisms, emerging cross-system interoperability protocols, and open research challenges. The review uses a PRISMA-based methodology, including literature from 2020 to March 2026, with particular focus on prominent frameworks such as LangChain, AutoGPT, AutoGen, MetaGPT, CAMEL, ChatDev, and CrewAI. The survey discusses the transition from modular chain-oriented to stateful graph-oriented and autonomous execution models and presents a systematic taxonomy of multi-agent collaboration architectures, including hierarchical, peer-to-peer, and role-based models. It also reviews the main internal communication mechanisms used to facilitate coordination and information sharing between LLM-based agents and distinguishes them from emerging interoperability protocols designed to support interaction across heterogeneous agents, tools, services, and frameworks. The analysis indicates that the promise of LLM-based agents for scalable automation, collaborative reasoning, and complex workflow execution comes with significant challenges in long-horizon reliability, evaluation standardization, communication security, cost-efficient orchestration, governance, and the interpretability of emergent multi-agent behavior. The survey covers architectural evolution, collaboration patterns, communication mechanisms, framework-level characteristics, and open research challenges. This provides a structured foundation for future research on reliable and trustworthy agentic artificial intelligence systems.
Unknown authors· Journal of Smart Algorithms...· 0 citations
This work introduces the concept of an agentic transaction and proposes an ACID-compliant agent system framework that reinterprets the classical ACID properties for agent execution through four semantic guarantees: Semantic Atomicity, Semantic Consistency, Semantic Isolation, and Semantic Durability.
: Integrating Large Language Models (LLMs) into production-grade business systems requires a clear boundary between deterministic application logic and non-deterministic AI behavior. This paper introduces a runtime architecture that extends the BAB (Business Application Builder) framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores. The core of the solution, the AgentRuntimeManager , moves beyond simple API wrappers by providing a robust environment for agent lifecycles, asynchronous event dispatching, and error handling through configurable retry policies. To ensure system stability, we decouple LLM providers (such as OpenAI and Ollama) using an abstraction layer built on LangChain4j, allowing for runtime provider switching without redeploying the core logic. We demonstrate the practical utility of this architecture through a self-healing log-monitoring chain. In this setup, agents intercept high-severity log events and, through a structured analysis service, propose corrective actions. A specialized SqlErrorHealingAgent illustrates how LLM-proposed fixes can be filtered through layered safety constraints, such as the automatic blocking of destructive SQL operations like DROP or TRUNCATE, before any action is committed. The result is an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either.
Borivoj Bogdanović, M. Segedinac, Z. Konjović et al.· SINTEZA· 0 citations
