Skip to content
Preprint

ViBe: Visual Behavior Adaptation for Perceptive Humanoid Whole-Body Control

Sep 2026 · 2 citations · 38 references
Computer Science

TL;DR

ViBe, a post-training framework for adapting motion trackers to perceptive control tasks, uses pre-trained visual encoders with a multi-query extractor module to learn task-relevant perceptive feedback and solves a goal-oriented Repose Cube task with a deliberately simple planner, demonstrating the efficacy of perceptive controllers, adapted by this approach.

Abstract

Motion tracking provides a scalable recipe for humanoid whole-body control. By design, the resulting trackers lack exteroceptive feedback hence reacting to the environment remains the responsibility of a higher-level planner. Existing perceptive controllers train geometry-only encoders from scratch, trading semantics for sim-to-real ease, and typically rely on teacher-student distillation for a task of interest. We present ViBe, a post-training framework for adapting motion trackers to perceptive control tasks. We leverage pre-trained visual encoders with a multi-query extractor module to learn task-relevant perceptive feedback. This feedback is grafted onto the tracker's input via low-rank adapters, enabling parameter-efficient fine-tuning. Given a task reward and a reference dataset, this modular controller can be adapted directly via policy optimization. Across four tasks, ViBe shows zero-shot sim-to-real transfer spanning perceptive walking on curbs and parkour, Repose Cube, omni-object loco-manipulation, and dodgeball, with visually robust performance across outdoor, low-light, and RGB distractor conditions. Finally, we solve a goal-oriented Repose Cube task with a deliberately simple planner, demonstrating the efficacy of perceptive controllers, adapted by our approach.

View source

Similar papers

Preprint Oct 2026

TRACE: Efficient Transfer Learning for Neural Whole-Body Control Across Humanoid Embodiments

TRACE first injects kinematic priors into input and output projection layers and aligns nominal joint dynamics to reduce embodiment mismatch and support stable fine-tuning, then combines encoder distillation with low-rank reinforcement learning on a compact, quality-screened motion set.

Qing-Yu Lan, Zhi-Hao Tian, Zi-Fan Wang et al. · 0 citations
Sep 2026

PASSAGE: Scaling Scene-Aligned Motion Learning for Perceptive Humanoid Traversal in Cluttered Environments

PASSAGE, a perception-conditioned planner--tracker framework for humanoid traversal, using virtual reality and inertial motion capture to collect 100 h of scene-aligned human motion across 1,500 cluttered scenes and tests across 50 unseen physical layouts demonstrate traversal without prebuilt maps or offboard computat...

Yuxuan Ma, Zi-Cheng Zeng, Chun-Li Peng et al. · 1 citation
Preprint Sep 2026

Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion Generators

General purpose humanoids require locomotion controllers that are multi-skill, perceptive, dynamic, and robust enough to go anywhere humans can. In this work, we present a two layer locomotion architecture: (1) a perceptive flow matching motion generator plans whole body trajectories from raw depth images while a (2) p...

Zachary Olkin, William D. Compton, Aaron D. Ames · 0 citations
Sep 2026

EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation

EgoAlign, a data-construction framework that converts these demonstrations into action and state supervision compatible with a general-purpose, continuous whole-body controller, without collecting physical-robot demonstrations, is presented.

Yi-Ming Jiang, Jin Chen, Chong-Yang Xu et al. · 0 citations
Preprint Sep 2026

Towards High-DoF Dexterous Manipulation through VLA Post-Training

A unified four-step post-training pipeline comprising a learned temporal hand-action codec, supervised fine-tuning, DAgger, and real-world residual reinforcement learning provides a practical path for adapting VLA foundation models to reliable real-world dexterous manipulation.

Jun-Lei Zhu, Shen-Zhe Yao, Chao-Gui Huang et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.