Skip to content

M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

Sep 2026 · 0 citations · 37 references
Computer Science

TL;DR

Experimental results demonstrate the proposed ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance, achieves superior reconstruction fidelity and significantly boosts the success rate of VLA models.

Abstract

Recent advancements have successfully adapted autoregressive language models to process multimodal signals, such as images and actions. Since raw action signals are continuous, effective tokenization is essential to map high-dimensional inputs into compact discrete tokens for autoregressive processing. However, existing discrete action tokenizers often suffer from high reconstruction loss, failing to preserve the fine-grained dynamics required for precise control. This"discretization bottleneck"significantly limits the performance ceiling of downstream Vision-Language-Action (VLA) models. To address this, we propose ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance. Our approach introduces two key structural innovations: (1) we decompose the latent action features into multiple heads, enabling the model to implicitly align specific heads with distinct action dimensions; (2) we assign independent codebooks to each head for quantization. By leveraging the combinatorial nature of multiple codebooks, we significantly expand the representational expressivity of the tokenizer, leading to substantially lower reconstruction loss compared to previous methods. We evaluate the ${M}^2$Tok-based VLA on the RoboTwin, Simpler-Env, and 3 zero-shot real-world tasks. Experimental results demonstrate our method not only achieves superior reconstruction fidelity but also significantly boosts the success rate of VLA models. Comprehensive ablation studies further confirm the effectiveness of the multi-head and multi-codebook mechanisms. Code is available at https://github.com/cpaaax/M2Tok.

View source

Similar papers

#machine learning Preprint Oct 2026

SUAVE: Unified Video-Action Models via Masked Diffusion

A single SUAVE model predicts long-horizon video and acts as a policy, competitive with dedicated world models and specialized action policies on static and dynamic manipulation tasks and shows that masked diffusion is a practical and versatile foundation for unified video-action modeling.

Rhythm Syed, Jean-Pierre Mercat, Sedrick Scott Keh et al. · 0 citations

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

This work introduces physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction, and presents ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization.

Shijie Lian, Bin Yu, Zhao-Long Shen et al. · 1 citation
Preprint Oct 2026

CHASE-VLA: Post-Training Quantization Framework for Vision-Language-Action Models with Chunk-Aware Scale Estimation

Vision-Language-Action (VLA) models map visual observations and language instructions to continuous robot actions, but a diffusion-based action expert (AE) poses a key challenge for low-bit post-training quantization (PTQ). The AE is repeatedly invoked across denoising steps and policy queries, where fixed calibration...

Jin Hyun, Jung Gyu Min, Gyuhyun Jung et al. · 0 citations
#artificial intelligence Preprint Oct 2026

TOAST: Stochastic Robot Action Tokenization for Autoregressive Vision-Language-Action Models

TOkenization of Action sequences with STochastic sampling (TOAST), a stochastic action tokenization method that samples alternative tokenizations of the same quantized action sequence during policy training, demonstrates the effectiveness of stochastic action tokenization for autoregressive robot policy learning, parti...

Keisuke Shirai, Tomohiro Motoda, Hanbit Oh et al. · 0 citations

Beyond Reconstruction Error: Analytical and Data-Driven Action Tokenization for Autoregressive Vision-Language-Action Models

It is shown that reconstruction fidelity alone cannot reliably select action representations for autoregressive control, motivating joint evaluation of geometric fidelity, sequence predictability, decoder stability, and closed-loop performance.

Yu-Xin Yang, Gao-Han He, Chang-Xue Guan et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.