Jun 2026· arXiv.org· Vol abs/2606.30555· 0 citations· 35 references
Computer Science
TL;DR
ANTAP (Automatic Non-Textual Agent Picker), an evaluation-driven routing architecture that discards indirect proxies in favor of active capability testing, is introduced, which distills performance into fixed behavioral operators within a shared semantic space.
Abstract
The rapid integration of Large Language Models (LLMs) has driven the evolution of Multi-Agent Systems (MAS), where specialized agents collaborate to execute complex workflows. Effective orchestration in these environments requires robust routing mechanisms to efficiently allocate tasks to the most suitable agent. However, existing routers fundamentally rely on unverified proxies, ranging from textual self-descriptions to static surrogate representations, to gauge an agent's competence. This reliance on non-empirical data creates a critical gap between an agent's projected profile and its actual operational capabilities, introducing severe security vulnerabilities. Malicious agents can easily misrepresent their proficiencies or harbor covert backdoors that evade both standard external analysis and static representation-learning techniques. In this work, we introduce ANTAP (Automatic Non-Textual Agent Picker), an evaluation-driven routing architecture that discards indirect proxies in favor of active capability testing. By dynamically querying agents to ascertain their true competencies empirically, ANTAP distills performance into fixed behavioral operators within a shared semantic space. At inference time, routing is performed via a purely non-textual algebraic projection, establishing a"linguistic firewall"that renders metadata-based attacks inexpressible. In our experiments, ANTAP achieves near-zero ASR against description-based injection attacks, compared to 67.3\% and above for the description-based router baseline. Against adaptive embedding attacks, ANTAP achieves substantially lower ASR than the embedding-based baseline, with a 20\% reduction, while remaining resilient to description manipulation by design.
AgentWeave is introduced, a deterministic pre-inference routing layer that constructs a bounded model-visible action space using eligibility, requirement, capability, and routing signals and shows lower mean local-model latency.
Saurav Singla, A. Singla, Advik Gupta et al.· 0 citations
Experimental results show that the proposed self-calibrating agentic framework successfully profiles the zero-knowledge workloads, achieving a higher accuracy than baseline LLM agents and establishing a robust foundation for deploying autonomous AI in decentralized infrastructures.
Fin Gentzen, Marla Grunewald, Iulisloi Zacarias et al.· 0 citations
Fixed-access network investigation remains labour-intensive because faults are expressed across heterogeneous evidence: customer reports, topology, alarms, optical measurements, device telemetry, configuration histories, tickets, software releases and operational knowledge. This review examines how retrieval-augmented generation (RAG) and agentic artificial intelligence can be combined to automate investigation without allowing a language model to substitute plausible narrative for verified engineering evidence. The study synthesises research published from 2020 to 2025 on retrieval, tool-using language agents, root-cause analysis, optical-access diagnostics, intent-based management and zero-touch operations, together with relevant fixed-access standards. It proposes an evidence-governed architecture in which a planning agent decomposes an incident, permission-aware retrievers assemble time-bounded context, specialised tools execute deterministic queries, a causal reasoning layer ranks hypotheses, and an independent verifier checks numerical results, provenance and policy compliance. The review argues that the most defensible role for generative models is orchestration and explanation, whereas calculations, state inspection and change execution should remain within typed, auditable tools. A staged workflow is developed for incident framing, evidence alignment, topology reconstruction, hypothesis testing, counterfactual checking, confidence calibration, human approval and organisational learning. Evaluation should therefore measure retrieval coverage, factual and numerical correctness, root-cause ranking, time-to-isolation, unsafe-action rate and operator trust rather than linguistic fluency alone. The resulting framework supports faster and more consistent diagnosis across fibre, copper, residential gateway and disaggregated access environments while preserving human authority over consequential network actions.
Rizwan Farooq Khan· Global academic journal of e...· 0 citations
Multi-agent systems built on large language models (LLMs) are increasingly deployed for complex tasks requiring autonomous planning, tool use, and inter-agent coordination. However, the non-deterministic nature of LLM outputs and the emergent behavior arising from agent interactions render traditional test oracles ineffective, creating a critical gap in quality assurance for agentic AI. This work introduces MORPHAGENT, a framework designed to address the oracle problem in multi-agent LLM systems through trace-based behavioral analysis. Our contributions are threefold: (1) goal-preservation relations that verify consistent goal achievement under input perturbations, (2) coordination-consistency relations that validate inter-agent delegation and communication patterns under agent substitution and reordering, and (3) tool-use integrity relations that ensure semantic equivalence of tool invocation sequences under prompt paraphrasing. MorphAgent instruments agent execution to capture structured traces comprising planning steps, tool calls, message exchanges, and final outputs, then systematically applies metamorphic transformations and checks behavioral invariants without requiring ground-truth oracles. We evaluate the framework on four multi-agent benchmarks spanning code generation, research synthesis, customer service, and data analysis tasks, encompassing 2,840 source-followup execution pairs across three LLM backends. Results show that MORPHAGENT detects 82.0% of seeded behavioral faults, including 90.3% of coordination failures and 81.7% of goal-deviation faults, while maintaining a false positive rate of 6.1%. The framework uncovers 14 previously unreported behavioral anomalies in established multi-agent frameworks, demonstrating its practical utility for assuring agentic AI reliability. These results suggest that trace-based metamorphic testing can serve as a practical foundation for reliable validation of emerging agentic AI systems.
Gopalakrishnan Marimuthu· International Conference on...· 0 citations
As autonomous agents rapidly evolve, their ability to reliably manipulate ubiquitous digital documents has become critical for enabling general-purpose AI assistants and automating complex workspace workflows. In this paper, we introduce DocOps, a deterministically verifiable evaluation framework underpinned by a hierarchical taxonomy that deconstructs document operations inspired by real-world practices into atomic dimensions and escalating workflow complexities. Based on DocOps, we systematically evaluate representative closed- and open-source models across various agentic harnesses, revealing that even the most advanced frontier configurations still exhibit profound limitations when handling highly coupled, long-range tasks. Furthermore, a fine-grained analysis of existing agents'manipulation behaviors uncovers 3 key failure modes: long-term state tracking collapse, shallow semantic verification, and destructive editing of structural metadata. Ultimately, our work exposes the capability boundaries of agents in maintaining global document consistency, shedding light on the future design of robust, non-destructive agents for complex digital ecosystems.
Jiazhen Jiang, Boxi Cao, Lingyong Yan et al.· 1 citation
The rapid development of large language models and multimodal large language models has accelerated the emergence of proactive agents capable of operating everyday tools and assisting users in real-world environments. However, existing benchmarks struggle to evaluate such agents effectively, as they often rely on sandboxed environments and single-turn evaluation paradigms. Moreover, their scenario-based task taxonomies mix multiple model capabilities within the same task category, making it difficult to identify the root causes of agent failures. To address these limitations, we introduce UniClawBench, the first capability-driven benchmark designed to evaluate proactive agents in dynamic, real-world settings. UniClawBench is built around five foundational model capabilities: Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, and Cross-Platform Coordination. Based on these capabilities, we design 400 bilingual real-world tasks. Unlike previous benchmarks that rely on static, pre-recorded answers, our benchmark evaluates agents in live Docker containers using fine-grained, step-by-step completion checkpoints. Furthermore, we design a closed-loop evaluation strategy comprising an executor agent, a hidden supervisor agent, and a user agent to simulate realistic multi-turn human feedback without leaking grading criteria. To disentangle base model capabilities from framework-level design choices, we evaluate state-of-the-art models under multiple agent frameworks. Through comprehensive comparisons across both models and frameworks, we show how base model capabilities and agent framework designs jointly shape performance in real-world environments. To facilitate future research, we make our benchmark and code publicly available at https://github.com/HKU-MMLab/UniClawBench.
Zhekai Chen, Chengqi Duan, Kaiyue Sun et al.· 1 citation