Aug 2026· Proceedings of the International Symposium on Combinatorial Search· 0 citations
Abstract
Actor-critic models are a class of model-free deep
reinforcement learning (RL) algorithms that have
demonstrated effectiveness across various robot learning
tasks. While considerable research has focused on improving
training stability and data sampling efficiency, most
deployment strategies have remained relatively simplistic,
typically relying on direct actor policy rollouts. In
contrast, we propose PACHS (Parallel Actor-Critic Heuristic
Search), an efficient parallel best-first search algorithm
for inference that leverages both components of the
actor-critic architecture: the actor network generates
actions, while the critic network provides cost-to-go
estimates to guide the search. Two levels of parallelism
are employed within the search---actions and cost-to-go
estimates are generated in batches by the actor and critic
networks respectively, and graph expansion is distributed
across multiple threads. We demonstrate the effectiveness
of our approach in robotic manipulation tasks, including
collision-free motion planning and contact-rich
interactions such as non-prehensile pushing. Visit
https://p-achs.github.io for demonstrations and examples.
A unified framework that combines centralized training with decentralized execution (CTDE) and a Hybrid Reward Architecture (HRA) is introduced that enables multiple actors to share a centralized multi-head critic and substantially improves both sample efficiency and policy performance.
Changhao Li, Yifang Zhang, Heng Zhang et al.· 0 citations
Group Relative Return Policy Optimization (GR2PO), a critic-free reinforcement learning framework for continuous robot control, is proposed and significantly outperforms critic-free baselines that use immediate rewards and performs competitively against state-of-the-art actor-critic methods.
Pengqin Wang, Qi-Ming Zhang, Shao-Jie Shen et al.· 0 citations
This work revisits the conventional offline RL paradigm and proposes decoupling policy improvement from actor training, and trains the actor solely to model the behavior distribution and perform policy improvement at inference time by reranking multiple actor-generated proposals with a separately learned critic.
Xu-Yao Lin, Yixiang Shan, Jin-Ru Duan et al.· 2 citations
An alternative reward shaping method (RS) is proposed that removes deceptive rewards at the expense of theoretical guarantees of PBRS, and another method named Locally-Guided Actor Critic (LG-AC) that rewards the agent for reaching intermediate goals achieves the best overall performance across tasks.
A Self-Evolutional single-agent/multi-agent Reinforcement Learning (SE-RL) framework that utilizes a Large Language Model (LLM) to design various RL algorithm modules, such as agent model design, reward function, profiling, communication, and state imagination, by leveraging the LLM generating module output or code.
Vincent Fu, Xin-Xin Xu, Weichen Xu et al.· Proceedings of the 32nd ACM...· 0 citations
Model-based reinforcement learning (MBRL) is a family of RL methods that learn a model of the environment and use it for action selection, making it well suited to robotics due to its sample efficiency. Combining learned models with online planning can further improve action selection, as the planner can exploit the mo...
Pietro Noah Crestaz, Mohamed Yassine Kabouri, Nicolas Mansard et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
