May 2026· arXiv.org· Vol abs/2605.30834· 8 citations· 95 references
Computer Science
TL;DR
This paper proposes Hide-and-Seek, a framework that formulates VLA failure detection as a coarsely supervised learning problem that achieves state-of-the-art multi-task failure detection performance with a practical accuracy--timeliness trade-off under conformal prediction, and generalizes well to both seen and unseen tasks.
Abstract
Vision-Language-Action (VLA) models enable robots to follow natural language instructions and generalize across diverse tasks, but they remain vulnerable to execution failures that compromise reliability in real-world deployment. Detecting such failures during execution is therefore critical for the robust deployment of embodied systems. Existing failure detection methods either rely on expensive action resampling or external models, while alternatives propagate trajectory-level labels uniformly across every timestep, obscuring localized failure signals. In this paper, we propose \textbf{Hide-and-Seek}, a framework that formulates VLA failure detection as a coarsely supervised learning problem. By combining inter-trajectory and intra-trajectory contrastive objectives, Hide-and-Seek localizes failure-indicative actions and induces temporally structured failure signals from trajectory-level supervision alone, without any step-level annotation. We evaluate Hide-and-Seek on LIBERO, VLABench, and a real-world robotic platform across three representative VLA policies: OpenVLA, $\pi_0$, and $\pi_{0.5}$.Our method achieves state-of-the-art multi-task failure detection performance with a practical accuracy--timeliness trade-off under conformal prediction, and generalizes well to both seen and unseen tasks.
Vision-language-action (VLA) policies have shown strong potential for general-purpose robotic manipulation, but they can still fail unpredictably during long-horizon execution, making reliable failure detection essential for safe deployment. Existing methods either rely on visual models that typically detect failures o...
Long-horizon robotic tasks are vulnerable to unexpected environmental changes that can render planned actions ineffective or unsafe. To address this, robots must detect such changes as they occur, interpret their impact, and adjust their actions accordingly. Traditional rule-based decision-making pipelines are brittle...
A. N. Ahmed, Omar Moured, Mughni Irfan Mohammed Abdul et al.· arXiv.org· 0 citations
Autonomous vehicles (AVs) operate in complex environments where failures are consequential. Sophisticated machine learning models for perception and planning are key to overcoming at least part of that complexity, but their black-box nature complicates validation and verification (V&V). The recent integration of Vision...
Tianwen Yu, Lu Feng, Sebastian G. Elbaum· 1 citation
VLA-Scope is introduced, a two-stage framework that combines input-shift characterization with execution history to predict failure during OOD rollouts and achieves a higher ROC-AUC than the evaluated ActProbe and SAFE-MLP baselines.
Results show that observable visual-proprioceptive-action history is sufficient to infer latent task states and enable practical failure recovery for existing VLA policies, and the proposed approach substantially improves closed-loop reliability under diverse perturbations.
Chang-Nan Song, Bin Qian, Yan Feng et al.· 0 citations
A Self-Evolving Embodied System (SEES) that learns from failures and improves the VLA policy without additional expert demonstrations, and can be integrated with different VLA backbones to progressively improve their long-horizon performance.
From feet to fingertips — we are teaching robots intelligent whole-body control, fine dexterity, and teamwork to complete a broad range of complex tasks.