Skip to content

Recovering Aggressively Pruned Vision-Language-Action Models with Offline Hidden-State Distillation

Sep 2026 · arXiv.org · Vol abs/2609.19579 · 0 citations · 29 references
Computer Science

TL;DR

Vision-language-action models let robots follow language instructions, but their language backbones of several billion parameters are the main obstacle to running them on robot hardware, and removing 63% of it from OpenVLA-OFT drops LIBERO-Long success from 93.2% to 0.8%.

Abstract

Vision-language-action (VLA) models let robots follow language instructions, but their language backbones of several billion parameters are the main obstacle to running them on robot hardware. Structured pruning reduces that backbone, and removing 63% of it from OpenVLA-OFT drops LIBERO-Long success from 93.2% to 0.8%. A recent approach restores such a model with supervised fine-tuning followed by reinforcement learning, which needs online rollouts and hundreds of GPU-hours. We recover most of the lost success entirely offline. Width pruning narrows the blocks but keeps the residual stream at its original size, so teacher and student hidden states have the same shape and are matched directly, without a projector. Training against a cache built in one teacher pass lifts the 63%-reduced student to within 3.5 points of the teacher in about 8 GPU-hours. A sweep over nine ratios locates where the recovery objective starts to matter. Up to 45% reduction the two do not differ significantly on OpenVLA-OFT. Hidden-state distillation then adds +2.1 to +4.5 points there between 63% and 87%, and +9.4 to +22.1 points on CogACT from 63% onward. At 81% on CogACT, a tripled recovery budget narrows the distilled student's gap to the teacher to 3.9 points on average, while supervised recovery stays more than 20 points below. At matched compression, width pruning yields higher success and depth pruning lower latency. On a 6-DoF manipulator, the distilled student at 72% reduction reaches 77.5% success against 59.5% for supervised recovery, runs 2.23x faster on-board than the teacher, and uses 62% less memory.

View source

Similar papers

Open access Sep 2026

Restoring the Right Stream: Training-Free OOD Robustness for Vision–Language–Action Policies

Gated Per-Stream Manifold Restoration (G-PSMR), a training-free wrapper for a frozen policy that instantiates the framework with entropy-gated visual consensus and gated relative-orientation debiasing, which preserves the within-episode orientation trajectory.

Zhi-Yin Yan, G. Shen · 0 citations
Preprint Aug 2026

Perturb the Thought, Not the Pixels: Latent-Space Rollout Diversification for Reinforcement Learning of Vision-Language Models

Noise-Contrastive GRPO is introduced, which injects scale-calibrated Gaussian noise into the last hidden layer of the prompt-encoding pass for half of each rollout group, branching those rollouts from a displaced departure state and integrates into a standard RLVR pipeline as a ~50-line change to the inference engine.

Michael M. Jerge, Joseph Pelczar, J. Downes · 1 citation

Reinforcement Learning for Real-Time Vision-Language-Action Policies

This work instantiates Real-Time EXPO-FT, an RL framework for finetuning real-time VLA policies that meets the real-time control requirements of dynamic real-world manipulation, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics.

Perry Dong, Kuo-Han Hung, D. Sadigh et al. · 2 citations
Sep 2026

Dense to MoE Adaptation for Compact Vision Language Action Policies

The results suggest that dense to MoE adaptation with dynamic expert deactivation is a practical direction for reducing active VLA model size without severe performance loss.

Mu-Chun Niu, Shuang Chen, Yu-Zhou Wu et al. · 0 citations

VLAQuantBench: Closed-Loop Evaluation of Post-Training Quantization for Vision-Language-Action Models

These results establish recipe-dependent interactions and identify concrete precision assignments, rather than universal layer-sensitivity rules, rather than universal layer-sensitivity rules for vision-language-action models.

Jiu-Yi Xu, Qing Jin, Mei-Da Chen et al. · 3 citations · ⚡1

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.