Skip to content
Preprint

H^2MARL: Hierarchical Heterogeneous Multi-agent Reinforcement Learning for Humanoid Robot Control

Nov 2026 · 0 citations · 25 references
Computer Science

TL;DR

This paper develops a closed-loop policy-executor co-optimization mechanism that jointly adapts the high-level pose planner and the low-level joint executor using shared interaction trajectories, thereby reducing the distribution mismatch between planning and execution.

Abstract

Coordinated whole-body control of humanoid robots remains challenging due to high-dimensional coupled dynamics and substantial physical heterogeneity across joint degrees of freedom (JDoFs). Existing multi-agent reinforcement learning methods either employ coarse agent partitions or directly apply generic heterogeneous-agent algorithms, while hierarchical controllers commonly optimize high-level planning and low-level execution independently, leading to uncoordinated motion. In this paper, we propose H^2MARL, a hierarchical heterogeneous multi-agent reinforcement learning framework with two key mechanisms. We first formulate humanoid whole-body control using a JDoF-level physical-heterogeneity-aware policy factorization, where each torque-actuated JDoF is conditioned on an explicit descriptor of its actuation, kinematic, and dynamic properties. Moreover, we develop a closed-loop policy-executor co-optimization mechanism that jointly adapts the high-level pose planner and the low-level joint executor using shared interaction trajectories, thereby reducing the distribution mismatch between planning and execution. Extensive experiments in MuJoCo and Isaac Gym demonstrate that H^2MARL consistently improves locomotion stability, gait coordination, and motion quality over homogeneous, coarse-grained, and independently optimized alternatives.

View source

Similar papers

#machine learning Preprint Oct 2026

MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blending

A permutation-based data augmentation strategy for homogeneous multi-humanoid systems, and theoretically show that the permuted samples preserve the policy-gradient direction of the original samples under the homogeneous Markov game formulation.

Yi-Fan Hu, Lu-Hang Hong, Ming-Kang Long et al. · 0 citations
Oct 2026

Enhancing heterogeneous multi-agent autonomous exploration in unstructured terrains via dual-layer collaborative control.

Multi-Agent Systems (MAS) are of great significance in real-world applications, as they can effectively manage complex tasks through decentralized collaboration. Most existing studies predominantly emphasize homogeneous MAS, i.e., systems consisting of agents with identical functional capabilities and standardized beha...

Ke-Xiang Zhao, Bing-Yu Hu, Ying-Ying Pan et al. · 0 citations
#machine learning Preprint Oct 2026

Exploiting Hierarchical Controller Structure in Contextual Parameter Learning for Humanoid Loco-Manipulation

This work proposes a contextual Bayesian optimization framework for joint parameter learning in hierarchical control systems, and evaluates the approach for humanoid loco-manipulation, jointly tuning a centroidal predictive controller and a whole-body controller for physical box pushing under varying box mass.

Sebastian Hirt, Lukas Theiner, Jan Peters et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Hierarchical Reinforcement Learning for Collision-Free Locomotion of an Underactuated Biped

A bipedal robot cannot deviate from its path to avoid an obstacle without disturbing its balance, and this coupling is most severe on underactuated platforms such as the biped considered here, which has four actuated joints per leg and no hip or ankle roll. This paper presents a Hierarchical Reinforcement Learning (HRL...

J. Sahoo, Saurabh Kumar, Surya Prakash S.K. et al. · 0 citations
Preprint Oct 2026

HA-DiPO: Diffusion Policy Optimization for Heterogeneous Cooperative Transport

Cooperative transport with heterogeneous legged robots requires coordinating different motion capabilities while supporting an unfixed shared payload. We present HA-DiPO (Heterogeneous-Agent Diffusion Policy Optimization), a hierarchical multi-agent reinforcement learning framework under centralized training with decen...

Xue-Bin Li, Jin-Peng Xiao, Zong Chen et al. · 0 citations

Cooperative Multi-Agent Vision-Language-Action Models via Reinforced Fine Tuning

A three-stage reinforced fine-tuning (RFT) pipeline for multi-agent VLAs is presented, which freezes the VLA and performs RL in its latent noise space and finds existing online RL for VLAs are less effective for hard multi-agent tasks, which are attributed to noisy co-exploration and unstable updates.

Rui-Xiao Xu, W. Kenny, Zhi-Qian Liu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.