Skip to content

Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World

Sep 2026 · 0 citations · 81 references
Computer Science

TL;DR

Open-UniMo, a unified Large Motion-Language Model (LMLM) trained on million-scale open-world motion-language data, and Open-MoBench, a VLM-guided benchmark for assessing text-to-motion (T2M) generation, motion-to-text (M2T) understanding, and bidirectional consistency are proposed.

Abstract

Unified motion generation and understanding is crucial for embodied AI systems that can both synthesize and interpret human actions in open-world environments. Existing motion-language models often treat motion as an auxiliary modality of a language model, leading to text-dominated representations and limited cross-modal interaction. Moreover, the next-token prediction paradigm is not naturally suited to long motion sequences, where autoregressive generation may accumulate prediction errors. To address these challenges, we propose Open-UniMo, a unified Large Motion-Language Model (LMLM) trained on million-scale open-world motion-language data. Open-UniMo promotes modality parity by extending Qwen's vocabulary of about 150K text tokens with 64K motion tokens, enabling motion and language to share a unified token space. We further introduce motion-consistent Chain-of-Thought reasoning as an intermediate representation to bridge language semantics and motion dynamics. Open-UniMo is trained with a two-stage pipeline, where supervised fine-tuning establishes CoT-guided bidirectional motion-language mapping and Group Relative Policy Optimization (GRPO) improves semantic alignment while mitigating cumulative errors in autoregressive motion-token generation. To support comprehensive evaluation, we propose Open-MoBench, a VLM-guided benchmark for assessing text-to-motion (T2M) generation, motion-to-text (M2T) understanding, and bidirectional consistency. Extensive experiments show that Open-UniMo achieves state-of-the-art performance on both conventional metrics and Open-MoBench. Furthermore, ablation studies reveal that M2T understanding is not primarily limited by motion-token vocabulary size; instead, coupling M2T with the learnable T2M generation path yields stronger cross-modal representations, demonstrating that generation can facilitate understanding in AR-based motion-language modeling.

View source

Similar papers

Preprint Oct 2026

UniWAM: Unified World-Action Model

UniWAM is introduced, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction and uncovers a log-linear scaling law of unified human-robot co-training.

Wen-Xuan Song, Jia-Yi Chen, Jing-Bo Wang et al. · 6 citations · ⚡2
Sep 2026

EgoMotion: Hierarchical Vision-Language Learning and Diffusion for Egocentric Motion Generation.

EgoMotion is proposed, a two-stage framework for vision-language-guided egocentric motion generation that achieves state-of-the-art performance and produces motion sequences that are both semantically grounded and kinematically superior to existing approaches.

Rui-Bing Hou, Ming-Yu Zhou, Yu-Wei Gui et al. · 0 citations
#computer vision Preprint Aug 2026

SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling

SeMoCo, a semantic-first motion codec, is introduced together with a dual-axis motion generator for language-conditioned motion generation and $\Omega$-MotionVerse, a large-scale, multi-source human-motion dataset unified under the SOMA representation is constructed.

Tianlv Huang, He-Tian Guo, Zi-Yi Cai et al. · 1 citation
Preprint Aug 2026

StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models

StreamPI is proposed, a streaming multimodal temporal modeling framework that equips single-frame VLA with temporal reasoning capability without introducing any additional parameters and seamlessly inherits pretrained single-frame weights and supports flexible single-frame and multi-frame inference.

Zhe Liu, Jing-Hua Hou, Yuxiang Lu et al. · 4 citations · ⚡1
Preprint Sep 2026

RoboSTAR: Next-Scale Autoregressive Sign Language Translation for Humanoid Robots

Sign-language interpretation in public communication relies on qualified professional interpreters and can be difficult to scale, motivating robotic signing as a complementary accessibility interface. We present RoBoSTAR, a text-conditioned sign language production (SLP) framework for generating human-centric sign moti...

Yu-Jia Zeng, Chensheng Peng, Yu-Xin Chen et al. · 0 citations
Sep 2026

MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence

MachEmbodied-U0, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture, is presented, a unified embodied foundation model connecting understanding and generation experts through a Mixture-of-Transformers architecture that combines competitive...

Hao-Ran Wen, Wen-Fu Wang, Kun-Song Shi et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.