TraJudger: Structure-Guided Failure Attribution for LLM-Powered Multi-Agent Systems
Abstract
LLM-powered multi-agent systems are increasingly deployed in production, yet diagnosing their failures remains a major operational challenge. Unlike failures in cloud architectures, which can often be diagnosed using structured diagnostic information, multi-agent system failures are embedded in unstructured natural-language interactions, cascade silently across semantic boundaries, and defy attribution due to a lack of prior domain knowledge. Existing LLM-as-a-Judge approaches treat execution trajectories holistically, misattributing cascaded failures and fabricating plausible but ungrounded explanations. To overcome the limitations of existing methods, we present TraJudger, a structure-guided failure attribution system deployed in production at a global top-tier online shopping service provider. TraJudger constructs a Hierarchical Cognitive Graph from raw trajectories to recover explicit inter-step dependencies, deploys four category-specific specialist agents for typed failure detection, and performs graph-constrained backward tracing to ground root-cause identification in verified dependency paths. On 121 production and 126 benchmark failure trajectories, TraJudger achieves a relative improvement of at least 15% and 13.3% over state-of-the-art methods at the agent and step levels. TraJudger has been deployed at \(\mathcal {C}ompany\) \(\mathcal {A}\) for over one month, reducing average time-to-attribution from 30 minutes to under one minute.