This work uses Sample-based Model Predictive Control entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets and validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies.
Abstract
Integrating locomotion and manipulation is essential for robot autonomy, but scaling standard Reinforcement Learning (RL) to complex tasks is severely bottlenecked by the slow, manual process of dense reward shaping. To bypass this limitation, we leverage Sample-based Model Predictive Control (SMPC) entirely in simulation as an automated, rapidly tunable expert to generate massive offline datasets. Because this data solves the fundamental exploration problem, we can train an off-policy RL agent using purely sparse task rewards, drastically reducing the time required to learn new skills and eliminating the need for manual tuning. Integrating this high-level agent with a low-level dynamic stability controller yields more optimal behaviors that strictly align with true task objectives, ultimately allowing the learned policies to surpass the original optimal control teacher. We validate the robustness of this sim-to-real framework by successfully deploying complex loco-manipulation skills across different morphologies, including an arm-equipped Spot quadruped and a G1 humanoid.
A unified framework that combines centralized training with decentralized execution (CTDE) and a Hybrid Reward Architecture (HRA) is introduced that enables multiple actors to share a centralized multi-head critic and substantially improves both sample efficiency and policy performance.
Changhao Li, Yifang Zhang, Heng Zhang et al.· 0 citations
Robotic manipulation policies trained via imitation learning, such as Action Chunking with Transformers (ACT), can achieve strong performance under ideal conditions but often remain sensitive to small execution errors and distribution shifts. Correcting these failures typically requires dataset aggregation and full-policy retraining, which is computationally expensive and unsuitable for real-time deployment. In this work, we propose Online Residual Policy Adaptation (ORPA), a framework that enables immediate, feedback-driven correction of robot actions without modifying the underlying policy parameters. ORPA augments a pretrained control policy with a lightweight, feedback-conditioned module that predicts residual adjustments directly in joint space, allowing the system to adapt its behavior at runtime. We evaluate ORPA on a set of precision-sensitive manipulation tasks using the ALOHA platform, demonstrating improvements in success rate and recovery from small perturbations compared to baseline control policies and rule-based inverse kinematics corrections.
Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai et al.· 0 citations
Hyper-redundant robots are essential for navigation in highly constrained environments, yet their high-dimensional kinematics impose a severe computational burden on real-time motion planning. While optimization-based methods ensure tracking precision, their high computational latency makes them unsuitable for online feedback loops; conversely, geometric heuristics offer speed but lack kinematic fidelity. To resolve this efficiency-accuracy trade-off, we present an imitation learning framework tailored for path-following tasks. First, to address the instability of expert data generation caused by non-differentiable minimax objectives, we propose a refined Soft-Maximum formulation that produces smooth, kinetically consistent demonstrations. Second, we mitigate the covariate shift inherent in Behavior Cloning (BC) through a two-stage noise-injection curriculum, enabling the agent to learn robust recovery policies entirely offline without requiring an interactive expert. Finally, we design a structured policy network that effectively fuses high-dimensional path descriptors with low-dimensional proprioceptive states. Extensive simulations demonstrate that our approach achieves optimization-level accuracy with inference speeds comparable to geometric heuristics, validating its efficacy for high-precision inspection tasks.
Nailong Bu, Ningyuan Luo, Chao-yuan Liu et al.· 2026 IEEE/ASME International...· 0 citations
This work identifies that additional research is still required to claim the successful resolution of the robotic arm reach-avoid task using DRL, and presents a comprehensive benchmark for the reachavoid task that accurately captures real-world complexities without simplifications.
Learning long-horizon robot manipulation remains difficult and time-consuming, especially under sparse rewards due to inefficient exploration and reward assignment. We present a minimal integration of large language models (LLMs) with reinforcement learning (RL) in which the LLM is used strictly as an online action proposer during early training to help with the RL agent. Given a task description and a compact scene abstraction, the LLM outputs a single parameterised primitive (pushing or grasping) that is validated by lightweight safety checks and mixed with policy actions via an epsilon-greedy scheduler. Besides, we study two simple policies that preserve the same low-level learner (grasping-biased model): (i) pushing-first reward shaping that temporarily increases the relative reward of pushing early in an episode, and (ii) a Proximal Policy Optimization (PPO) high-level switch that explicitly selects between push and grasp while Soft Actor-Critic (SAC) handles low-level parameterisation. In simulation, LLM-guided exploration accelerates early learning, and the proposed methods help further mitigate grasping overuse in the cluttered environment. Besides, the LLM-guided RL agent could achieve a higher success rate at 89.5% compared with the baseline method at 68%.
Meiyuan Gong, Yan Gao, Ze Ji· 2026 IEEE International Conf...· 0 citations
This paper proposes a nested kino-dynamic framework for rapid feasibility checking and dynamically consistent trajectory generation given a candidate contact sequence and shows that the generated trajectories can be tracked using a reinforcement learning (RL)-based controller and are of sufficiently high quality for execution in real-world loco-manipulation scenarios.
Michal Ciebielski, Shafeef Omar, Aaron M. Johnson et al.· 0 citations
