SteerQuant, a 4-bit quantization framework for WAMs that steers errors toward computations with less influence on final actions, is introduced and Rudder, a 4-bit inference engine for WAMs that fuses scaling and output compensation into low-bit kernels is developed.
Abstract
World-action models (WAMs) jointly generate future world states and actions through iterative denoising, using shared weights to process heterogeneous semantic streams of video, proprioceptive, and action tokens. Quantization reduces inference cost, but comparable numerical errors in different streams can have markedly different effects on final actions, making numerical accuracy alone insufficient for reliable control. We introduce SteerQuant, a 4-bit quantization framework for WAMs that steers errors toward computations with less influence on final actions. It maps how each stream's quantization errors affect final actions and uses this map to guide shared channel scaling. Activation scaling is further calibrated for each stream and denoising step to accommodate changes in activation ranges and action impact. This adapts quantization to different stream requirements without duplicating weights or increasing bit-widths for selected streams. To reduce the extra kernel launches and memory traffic introduced by scaling, we develop Rudder, a 4-bit inference engine for WAMs that fuses scaling and output compensation into low-bit kernels. Under W4A8 and W4A4, SteerQuant maintains mean LIBERO success within 0.8 percentage points of full precision, while delivering up to $2.23\times$ denoising speedup over BF16 across three WAMs with reduced peak GPU memory usage. On a real dual-arm robot, W4A8 deployment achieves a $1.35\times$ end-to-end inference speedup while maintaining average task success relative to BF16.
World Action Models (WAMs) jointly generate video and robot actions through iterative diffusion and perform strongly in robotic manipulation. However, their prohibitive compute and memory costs pose substantial deployment challenges. Post-training quantization (PTQ) can reduce these costs, but existing PTQ methods such...
Arash Akbari, Arman Akbari, Jing Luo et al.· arXiv.org· 0 citations
AnyStep World Action Model is introduced, a general framework for tunable-budget prediction and scene-dependent computation allocation and a lightweight risk-benefit scheduler predicts teacher-curvature-based difficulty and budget-specific student-teacher fidelity from a single one-step preview, selecting the smallest...
Rui Wang, Xiang-Yuan Wang, Dong-Lin Yang et al.· arXiv.org· 0 citations
World Action Models (WAMs) have emerged as a promising alternative for embodied intelligence by jointly predicting future world states and robot actions. However, their heavy visual encoding and iterative generative inference incur substantial latency, limiting their applicability to high-frequency closed-loop control....
Zeng-Yuan Zuo, Yong-Chang Zhang, Guang-Yu Zhao et al.· 0 citations
Action-Guided Quantization is proposed, which combines action sensitivity with a measured proxy for weight recovery cost to allocate matrix bitwidths under storage budgets to allocate matrix bitwidths under storage budgets.
Xu-Hong Qiu, Zhuoyu Wu, Qi Zheng et al.· 0 citations
Post-training quantization (PTQ) reduces the memory footprint of vision-language-action (VLA) models, but lowering bit widths while maintaining task performance remains challenging. Existing work has used sensitivity estimates from calibration data to assign bit widths to different model components. However, using task...
Jiu-Yi Xu, Xiao Hu, Mei-Da Chen et al.· 0 citations
Quantization lowers deployment costs of vision-language-action (VLA) policies but can compromise task completion. Across five VLA models and 4,050 real-world trials, we systematically reveal that quantization-aware training (QAT) generally improves on post-training quantization (PTQ) but leaves substantial gaps to full...
Seongmin Park, Jaewon Lee, Yujung Heo et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.