Skip to content

Audit-First VAPO: Risk-Certified Selective Updates under Imperfect Verification

Sep 2026 · arXiv.org · Vol abs/2609.33662 · 0 citations · 27 references
Computer Science

TL;DR

This work introduces Audit-First VAPO, which separates discrete directional admission from continuous magnitude control, and evaluates two models on two reasoning benchmarks against static RLVR, matched-random selection, confidence thresholding, noise correction, and verifier augmentation.

Abstract

Imperfect verifiers can assign a harmful update direction even when clipping and regularization bound its magnitude. We introduce Audit-First VAPO, which separates discrete directional admission from continuous magnitude control. An observation-only accept-appeal-abstain policy uses a finite secondary-verification budget; its action trace is frozen before clean labels are joined. Simultaneous finite-sample bounds then certify selected harmful risk, coverage, and verifier-call rate over a predeclared policy family. Conditional Hoeffding-Azuma bounds account for the dependence induced by shared budgets, and rollout or verifier changes initiate a new certification stage. After admission, a bounded trust-clip-KL actuator controls magnitude. We evaluate two models on two reasoning benchmarks against static RLVR, matched-random selection, confidence thresholding, noise correction, and verifier augmentation. On Qwen3.5-0.8B and GSM8K at target risk $\rho=0.08$, RC-VAPO achieves 74.1% accuracy, selected harmful risk 0.0697, coverage 0.4125, and relative verifier cost $1.16\times$. At matched coverage and update magnitude, its selected-risk difference from matched random is -0.0260 with paired 95% interval $[-0.0364,-0.0157]$. Across asymmetric, confidence-dependent, and correlated-verifier noise, the certificate is satisfied on 57 of 60 independent runs. These comparisons isolate informative directional selection from proposal suppression, update shrinkage, and additional verifier computation.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents

This work argues that update admission must be assessed through both error control and retained learning opportunities at a stated interaction budget, and identifies a concrete failure: a range-based confidence gate cannot certify unchanged old-task behavior within otherwise substantial budgets.

Qin-Zhen Ma, Rui-Hai Wu · 0 citations
Preprint Aug 2026

CFR without Unbiasedness: Deterministic Guarantees for Persistent Public-Chance Schedules

A deterministic target-transfer theorem is established for uniform, nonnested additive public cuts that bounds full-cut exploitability by regret on the delivered feedback and a public-debit term that couples prefix coverage discrepancy with motion along the realized strategy path.

Jiaxing Guo, Lei Ye · 0 citations
#artificial intelligence Preprint Sep 2026

Deposon: An Auditable, Conservation-Guaranteed, Game-Theoretically Tested Scattering Layer over LLM Reasoning Paths

Multi-step LLM reasoning lacks a machine-recheckable ledger: discarded reasoning paths leave no auditable record. We propose the Deposon scattering layer, which binds each node of an LLM-generated concept-decomposition graph to a two-parameter Deposon state; paths undergo three-channel scattering -- transmission, refle...

Qi-Hao Yuan · 0 citations
Preprint Open access Sep 2026

From Public Evidence to Contractual Outcome: First and Stable Decidability on Kalshi

Public evidence can become sufficient to settle a prediction-market contract before the venue records its first determination, but the relevant boundary depends on the applicable rule version, exact release object, source hierarchy, correction history, and unfinished contract conditions. This paper defines two Kalshi c...

M. Nechepurenko · 3 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.