IDV: Customer-Context-Aware Multi-Engine Architecture for AI-Agent Intent Drift Detection in Finance

Abstract

AI agents operating on customer infrastructure in financial services face a governance challenge: customer intent specifications are category-level ("customer accounts," "process payments") rather than instance-level (specific account IDs, segregation-of-duties constraints). We empirically establish that this ambiguity produces measurable interpretation divergence at the LLM layer: two frontier LLMs (GPT-5.5, Claude Opus 4.7) labeling the same naturalistic agent traces under identical Intent Anchor Objects (IAOs) disagree on drift_detected for 30% of cases. When one judge labels a trace as instance-level entity access (Object-type drift), the inter-judge disagreement rate is 82.4% [95% CI 65–93%], a 2.7× enrichment relative to the marginal Object label rate. This suggests that single-pass LLM-only monitoring is unstable as a sole arbiter for category-level IAO boundary cases under the tested prompting strategy. We present IDV (Intent Drift Verification), a multi-engine architecture that runs four parallel detection engines (Authority, Coherence, Scope, Impact) on agent events with a two-layer adjudicator preserving the invariant drift_detected independent of Impact. On IDV-1000 (850 author-constructed synthetic + 150 naturalistic traces), IDV achieves F1=93.9% (precision 95.9%, recall 92.1%) and AUROC 98.7% on the author-injected subset, at sub-300ms end-to-end latency. Against full N=850 LLM-Judge baselines under JSON-only prompting, IDV achieves a 6.5× reduction in false-positive count (22 FP vs. 143/140 for GPT-5.5/GPT-4o, at precision 79.5%/79.9%). We identify system-design directions that customer-context-aware monitoring naturally supports: interactive IAO refinement, rolling-baseline IAO learning, and ambiguity-aware response routing.