Skip to content

Who Holds the Pen? Let Specifications, Not Agents, Sign Off

Sep 2026 · arXiv.org · Vol abs/2609.29921 · 0 citations · 25 references
Computer Science

TL;DR

Experiments on guideline-following and artifact-generation tasks show that specifications can serve not merely as behavioral guidance, but as authority over compliant execution and completion.

Abstract

Large language model agents increasingly combine generation, decision-making, execution, and self-evaluation within a single agentic loop. Although they operate under external specifications such as task instructions, guidelines, output schemas, and reusable skills, these specifications typically remain context for the same model that acts and declares completion, leaving no independent specification authority boundary. We identify two resulting gaps. The understanding--execution gap arises when a requirement is understood but not satisfied in execution; the state--authority gap arises when an agent's interpretation or completion claim does not establish the required state. On SkillsBench, using only agent-visible prompts, workspace information, and injected skill specifications, we extract 509 source-grounded task directions. Across seven models, only 79.6%--86.4% are satisfied, while completion-claim rates exceed official evaluator pass rates by 28.7--37.9 percentage points. We therefore separate agent proposals from authoritative state. Agents may plan, act, and request completion, but only admissible evidence from qualified providers may establish specification-governed state. SpecHarness operationalizes this principle by compiling visible specifications into source-linked obligations and governing execution and finalization through versioned obligation state. Verifiable requirements are mediated or validated at runtime, while ambiguous or subjective requirements remain advisory. Experiments on guideline-following and artifact-generation tasks show that specifications can serve not merely as behavioral guidance, but as authority over compliant execution and completion.

View source

Similar papers

Dude, Where's My State? Execution Information Requirements for Stateful Agents

The EIR is introduced, a lower bound on the information that must remain accessible for correct completion under specified task and access conditions and LACUNA is developed, a framework that generates tasks with known dependencies and varies information demand, retention, and recovery separately from the difficulty of...

N. Mehrotra, Ashish Tiwari, Priyanshu Gupta et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Substrate-Aware AI Agents: Execution Context as a First-Class Input

A minimal execution contract induces proactive structural adaptation in generated programs, shifting computation away from unconstrained allocations and substantially improving observed resource-time profiles before execution, establishing a controlled proof of concept for substrate-aware agent planning.

Manu Agrawal · 0 citations
Review Aug 2026

Evaluating Skills, Not Just Agents: Agentic Continuous Evaluation of Skills

ACES (Agentic Continuous Evaluation of Skills), a repository-native framework for evaluating skills and product capability packages as executable agent artifacts, is presented.

Christopher Kevin, Narendran Raghavan, J. Puget et al. · 3 citations · ⚡1

How Strongly Should Task State Influence an LLM Agent?

Across three models, two reasoning regimes, and two domains, four findings hold without per-turn reasoning: displaying accurate state is unreliable, an unverified ledger the agent writes itself beats an accurate checklist it is shown, directives help in proportion to the model's obedience, and enforcement needs no obed...

C. Zhang, Wonbin Kweon, Jia-Wei Han · 0 citations

SINGED: Correct Outputs Do Not Certify Safe Execution in LLM Agents

This work introduces SINGED (Source Integrity and the Nonidentifiability Gap in Execution Decisions for LLM Agents), a controlled benchmark covering five primary and two held-out task families and exposes a rank-, evidence-, and choice-sensitive outcome-to-execution gap.

XiaoYu Xu, Zi Liang, Min-Xin Du et al. · 0 citations
Preprint Aug 2026

When May an Agent Stop? Evidence-Carrying Termination for Tool-Using LLMs

Evidence-Carrying Termination (ECT): an agent may return COMPLETE only when a typed certificate binds every required answer claim to valid, in-scope trace evidence and a deterministic replay reconstructs the claimed value.

Jason Liu · 3 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.