Direction-Scale Decomposition (DSD), an action representation that decomposes translation and rotation increments into direction and scale components before tokenization, is introduced, suggesting its potential to mitigate performance degradation when training on large and diverse dataset mixtures.
Abstract
Action representation plays a central role in discrete-token vision-language-action (VLA) learning but remains underexamined. Under conventional pose-increment representations, action tokens are sensitive to execution speed and dataset-specific normalization, potentially obscuring geometric structure shared across demonstrations and datasets. We introduce Direction-Scale Decomposition (DSD), an action representation that decomposes translation and rotation increments into direction and scale components before tokenization. DSD isolates motion direction while retaining magnitudes in separate scale channels. We evaluate DSD with uniform binning (BIN) and BEAST, a B-spline-based tokenizer, in simulation and real-world manipulation under both single-dataset and mixed-dataset training. On LIBERO, DSD improves average success rates with both tokenizers. On SimplerEnv, DSD-BIN outperforms BIN by 10.3 percentage points in overall success rate under mixed-dataset training. Real-robot experiments further show gains both with and without robotics pretraining. These results support DSD as an effective action representation for discrete-token VLA models and suggest its potential to mitigate performance degradation when training on large and diverse dataset mixtures. Our project page with additional resources is available at https://vla-dsd.github.io/
This work introduces physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction, and presents ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization.
Shijie Lian, Bin Yu, Zhao-Long Shen et al.· arXiv.org· 1 citation
This work proposes VLAct, a VLA-oriented VLM backbone trained on broad, heterogeneous, multi-embodiment robot data before task-specific fine-tuning, showing that representation-centric continued pre-training can deliver highly competitive performance under a modest compute budget and is an important independent axis of...
Sen-Qiao Yang, Cheng-Yao Wang, Yu-Xin Chen et al.· 8 citations
SALT is introduced, a Semantically ALigned action Tokenizer that augments a VQ-VAE-style tokenizer with an auxiliary objective requiring a frozen vision-language model to recover the episode instruction from quantized action latents to substantially improve language-conditioned control.
Wen-Jie Li, Yash Jangir, Ignacy Stepka et al.· 1 citation
The real-robot benchmark demonstrates that StellaVLA can use both human/robot demos and human-to-robot (XR) demos as in-context structured demonstration to help VLA model adapt to OOD tasks.
Siyu Xu, Yun-Ke Wang, Zi-Jian Wang et al.· 3 citations
UniWAM is introduced, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction and uncovers a log-linear scaling law of unified human-robot co-training.
Wen-Xuan Song, Jia-Yi Chen, Jing-Bo Wang et al.· 6 citations· ⚡2
Time-Frequency Geometric Cross-Attention (TFGCA), a drop-in module repairing both blind spots in vision-language-action policies, uses a per-dimension learnable stationary wavelet transform to decompose the action chunk into time-frequency tokens, and each time token retrieves information from them via a cross-attentio...
Sheng-Ye Dong, Hao-Chen Niu, Hao Liu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.