These results show that compiling procedural structure improves the reliability and efficiency of skill execution while retaining model judgment where it is needed, and shows that compiling procedural structure improves the reliability and efficiency of skill execution while retaining model judgment where it is needed.
Abstract
Agent skills provide a reusable way to specify multi-step agent behavior, but they remain natural-language specifications interpreted by the model at runtime. As a result, required tool calls, ordering constraints, and checks may be skipped even when explicitly prescribed by the skill. We introduce Skill Compilation, a paradigm that translates natural-language skills into executable agent programs while preserving model judgment where semantic decisions are required. We realize this idea in SIGIL. SIGIL extracts source-grounded requirements, decomposes them using a closed Agent Instruction Set (AIS), composes them into AG-IR with explicit ownership, data flow, and control flow, and deterministically lowers validated AG-IR into executable code. Across 33 publicly available SKILL.md files and three runtime models, SIGIL increases mean Applicable-Mandate Compliance (AMC), the fraction of applicable skill requirements satisfied during execution, from 66.0% with direct skill execution to 88.6%. SIGIL also reduces total runtime token consumption by 2.40-5.95x. These results show that compiling procedural structure improves the reliability and efficiency of skill execution while retaining model judgment where it is needed.
Skill Compilation is introduced, realized in SIGIL, which compiles a prose skill into an executable harness, and is model-independent: the harness holds at 86% across two model generations while prose swings from 56% to 68%.
Jayanaka L. Dantanarayana, Savini Kashmira, Lingjia Tang et al.· 0 citations
Agent Skills can specify procedural and resource obligations for tool use, and language models instantiate them as concrete programs. However, when models turn this guidance into code for existing tool interfaces, even a semantically correct program may load an entire input and exceed the memory available to one tool call. We present SkillEffect, a checked-lowering runtime for computations with a recoverable source relation, an audited bounded implementation, and a registered output postcondition. Before granting execution authority, an independent checker rebuilds each proposed lowering from the submitted program and immutable input. Every relation plugin supplies a source recognizer, input-fact extractor, bounded-IR constructor, arena-bound function, and postcondition; one common runtime provides checked selection, bounded-VM execution, atomic capacity leasing, and staged publication. Generality in SkillEffect is architectural rather than automatic: each supported computation requires an audited relation plugin, while the dispatch, resource-control, execution, and publication mechanisms are shared across plugins. Across six operator families, bounded access substantially reduces peak memory and improves completion under externally fixed caps. Six plugins instantiate the same contract across five execution patterns, from streaming reduction to bounded-heap Top-k. The XLSX onboarding study and Top-k extension show that a new relation and a new retained-state pattern reuse the same trust boundary, while the checker accepts all evaluated legal configurations and rejects all adversarial proposals. Together, these results show that one checked-lowering architecture can enforce heterogeneous registered memory relations at Agent tool dispatch.
CEDAR is presented, a counterexample-guided framework that grounds instructions as regular languages over environment event traces and represents both skills and specifications as deterministic finite automata, suggesting that regular languages offer a practical verification layer between natural-language instructions and embodied-agent policies.
Le Chen, Alvaro Velasquez, Ashutosh Trivedi· 0 citations
This work introduces SkillLogic, a framework for analyzing logical relations in skill files and constructing executable tests from them, and establishes logical-relation following as a distinct reliability challenge for skill-guided agents.
Xuan Chen, Chengpeng Wang, Lu Yan et al.· 0 citations
COVENANT is proposed, a compiler-and-interpreter architecture for workflow-aligned agent execution that substantially mitigates workflow misalignment, moving LLM-agent alignment beyond isolated prompt following toward reliable execution of complex and multi-step workflows.
Multi-agent systems built on large language models (LLMs) are increasingly deployed for complex tasks requiring autonomous planning, tool use, and inter-agent coordination. However, the non-deterministic nature of LLM outputs and the emergent behavior arising from agent interactions render traditional test oracles ineffective, creating a critical gap in quality assurance for agentic AI. This work introduces MORPHAGENT, a framework designed to address the oracle problem in multi-agent LLM systems through trace-based behavioral analysis. Our contributions are threefold: (1) goal-preservation relations that verify consistent goal achievement under input perturbations, (2) coordination-consistency relations that validate inter-agent delegation and communication patterns under agent substitution and reordering, and (3) tool-use integrity relations that ensure semantic equivalence of tool invocation sequences under prompt paraphrasing. MorphAgent instruments agent execution to capture structured traces comprising planning steps, tool calls, message exchanges, and final outputs, then systematically applies metamorphic transformations and checks behavioral invariants without requiring ground-truth oracles. We evaluate the framework on four multi-agent benchmarks spanning code generation, research synthesis, customer service, and data analysis tasks, encompassing 2,840 source-followup execution pairs across three LLM backends. Results show that MORPHAGENT detects 82.0% of seeded behavioral faults, including 90.3% of coordination failures and 81.7% of goal-deviation faults, while maintaining a false positive rate of 6.1%. The framework uncovers 14 previously unreported behavioral anomalies in established multi-agent frameworks, demonstrating its practical utility for assuring agentic AI reliability. These results suggest that trace-based metamorphic testing can serve as a practical foundation for reliable validation of emerging agentic AI systems.
Gopalakrishnan Marimuthu· International Conference on...· 0 citations
