This work developed AgentInspect, a framework that automatically detects six types of behavioral failures in LangChain-based AI agents by analyzing their execution trajectories across three evaluation settings: a baseline setting using real tool responses, a simulated setting incorporating synthetic tool responses, and a hybrid setting that combines the real and simulated tool responses.
This work presents ATLAS (Automata Learning for Agent Trajectory Analysis and Strategy Discovery), an approach for recovering interpretable behavioral models from agent trajectories that enable systematic understanding and analysis of otherwise opaque AI agents.
Ignacio D. Lopez-Miguel, A. Happe, J. Cito et al.· Proceedings of the ACM/IEEE...· 1 citation
This work shows that integrating structured abstractions with LLM-guided reasoning enables effective, reliable, and interpretable diagnosis for agent failures, and introduces the concept of neural invariants to specify agent behavior properties.
Jia-Yi Bi, Yanjie Gao, Yuan-Min Xie et al.· 0 citations
We propose a formal model for testing AI agents in stateful environments, using Finite State Machine (FSM) semantics to enable deterministic, trace-level evaluation via an explicit test oracle. Our approach introduces a classified action alphabet and six computable failure-mode detectors for automatic analysis over age...
Ahmed Twabi, Yepeng Ding, Tohru Kondo· International Conference on...· 0 citations
This work presents a post-hoc XAI framework that transforms a lengthy agent's execution trace into a structured report and a faithful natural-language explanation explicitly grounded in its observable behavior, outperforming naive LLM-generated explanations.
Vittoria Vineis, Fabiano Veglianti, Lorenzo Antonelli et al.· 0 citations
This survey synthesizes 257 papers spanning agent evaluation, software assurance, cyber-physical systems, runtime monitoring, and regulatory guidance in order to characterize the validation problem for agentic systems, and concludes with a lifecycle-oriented research agenda centered on bounded-autonomy specifications,...
Fabio Orazio Mirto, L. D'Agati, Giuseppe Tricomi et al.· arXiv.org· 0 citations
AEROBAT is introduced, the first multi-agent system to automate behavioral scientific research on AI agents and demonstrates that automated behavioral scientific research on AI agents can complement and extend the reach of manual research.
S. Lee, Jongha Lee, Jaewan Chun et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
