2026· Poster Volume 0008 The 2026 Twenty-Second International Conference on Intelligent Computing July 23-26, 2026 Toronto, Canada· 0 citations
TL;DR
This study cautions against transplanting verification into grounding pipelines and identifies calibrated abstention as a property worth preserving and proposes an abstention-aware verifier that intervenes only under sufficient candidate coverage and confidence.
Abstract
Web agents built on large vision-language models (VLMs) increasingly adopt a two-stage design: a grounding stage proposes candidate elements on a page, and an action stage decides which element to operate on and how. A natural way to strengthen such agents is to insert a pre-action verifier that re-scores the grounded candidates before acting, echoing the gains that verification and self-refinement bring to language-model reasoning. We test this assumption on the Mind2Web benchmark and report a counter-intuitive result: a GLM-4.6V pre-action verifier does not help and in fact degrades performance, low-ering the action-level step success rate from 34.8% to 23.7% on our evalua-tion subset. Through a step-level analysis we attribute this degradation to two causes. First, the offline multiple-choice protocol has limited candidate cov-erage, as the gold element is absent from the candidate set in roughly 80% of steps, so most steps are unsolvable regardless of verification. Second, and more decisively, the verifier mis-ranks candidates on the solvable steps and discards the grounding stage's calibrated abstention on the unsolvable majori-ty, so it removes a safe default without improving accuracy: it wins 5 steps but loses 20. Guided by this diagnosis, we propose an abstention-aware veri-fier that intervenes only under sufficient candidate coverage and confidence. Our study cautions against transplanting verification into grounding pipelines and identifies calibrated abstention as a property worth preserving.
A pipeline promoting an AI system publishes records claiming the thing evaluated is the thing deployed and that the evidence licensed the transition, and measures whether those records can express that claim and whether it holds where declared.
Rebuild-dossier is presented, an open-source tool that locks an application's real interface - its exact inputs and outputs - before any code is written, then enforces one-test-at-a-time building through automated checks, not written instructions alone.
The ORCA-bench benchmark is introduced, a benchmark that puts general-purpose coding agents in a production-fidelity oncall setting and is a lower bound on the engineering investment required before frontier coding agents can be safely entrusted with production reliability.
Albert Gong, Kyuseong Choi, Abhineet Agarwal et al.· 0 citations
How do you verify a long-horizon agent when its own state and self-reports are exactly what you cannot trust? We present an agent instrument built so that verification is structural rather than post-hoc. A deterministic Executive owns all belief; a language model may only file typed proposals, and a claim is admitted only when a prediction pre-registered before acting is matched against observation by code. Two properties make the instrument a verifier of its own science, not just of the agent: every run invalidates itself when per-organ write-error, render-size, or salted-canary-echo floors are breached (four of the first eight architecture runs were invalidated, each localizing a real defect); and a render-invisible shadow reference compiles the plan the full system would have committed in every ablation cell, so drift metrics are defined even where the mechanism under test has been removed. Using this instrument we report a clean, single-variable result on a failure every long-horizon agent suffers: ablating the commitment mechanism flips goal-abandonment from 0.00 to 1.00 while binding error stays flat at 0.00 (three seeds per cell, up to 394 reference beats per run, every run gated valid). The binding channel, by contrast, does not reappear as per-beat drift when its repair is ablated -- because binding is code-owned, the failure class is structurally absorbed, its only residue appearing one layer upstream as a collapse in hypothesis formation. We report these under full disclosure that task efficacy is null (zero level completions across 52 gated runs on ARC-AGI-3), pre-registered as a structural defeater. The contribution is a verification methodology for agent development and the drift decomposition it makes measurable.
ClawTrack is presented, a dual-assessment benchmark that simultaneously measures what an agent achieves (Task Score) and how it achieves it (Process Score) and finds that process scores effectively attribute success and failure to specific reasoning dimensions, filtering lucky passes invisible to outcome-only evaluation.
Xingjian Wu, Xuhan Zhu, Xingchen Liu et al.· 0 citations
NIS-Agent is proposed, which applies context isolation at the two decision points most vulnerable to inertia bias: webpage triage and final-answer validation, and trains an 8B model to be intrinsically more resistant to inertia bias.
Xiangdong Zhang, Zhanwei Zhang, Zhihang Fu et al.· 0 citations
