Targeted and Traceable Investigation of Multi-Agent LLM Dialogue via Semantic Bundling of Knowledge Graphs
TL;DR
This work presents a targeted and traceable approach to investigating multi-agent LLM dialogue, applied to the VAST Challenge 2026 MC1 dataset, and finds that the KG structure enables users to identify actors worth investigating faster and summarizing only the region surrounding an actor of interest better supports per-actor attribution than reading raw conversations.
Abstract
Multi-agent LLM systems today are increasingly automated, logging LLM-LLM interactions as conversational transcripts. Yet analyzing such dialogue for insights remains challenging, including attributing behaviors to the correct actor and summarizing interactions across a long exchange. We present a targeted and traceable approach to investigating multi-agent LLM dialogue, applied to the VAST Challenge 2026 MC1 dataset. The challenge asks participants to reconstruct and explain which internal communications among AI agents at TenantThread, a property tech company, led to an inappropriate information release. We first convert the dialogue into a knowledge graph (KG) and then investigate it with AgentK, a visual analytics system for interactive Semantic Bundling of nodes and edges. We found that our approach directly addresses two main challenges: (1) the KG structure enables users to identify actors worth investigating faster; and (2) summarizing only the region surrounding an actor of interest better supports per-actor attribution than reading raw conversations.