This work identifies a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach, and empirically demonstrates that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance.
Abstract
Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.
Differentiable simulation enables efficient policy optimization through analytic gradients, yet safety is commonly incorporated as a weighted penalty, requiring manual tuning of the trade-off between task performance and constraint satisfaction. We instead formulate obstacle avoidance as an explicit constraint and trai...
Yan-Rui Tan, Praveen Krishnapur, Ze-Yu Feng et al.· 0 citations
This article addresses the critical reliability and safety challenges in multistation multirobot welding task allocation for automated manufacturing, where conventional task assignment approaches often overlook physical collision risks, leading to system failures and production downtime. Unlike existing methods that ei...
Chao-Xu Mu, Jun-Nan Zhang, Ke Wang et al.· IEEE Transactions on Reliabi...· 0 citations
Safe navigation in human crowds requires efficiency and reliability. Learning-based policies run fast but can fail under distribution shift, whereas Hamilton–Jacobi (HJ) reachability offers principled safety guarantees but is computationally expensive. We propose a prediction-conditioned reachability framework that bri...
I. Winata, Junghyun Oh· IEEE Robotics and Automation...· 0 citations
Constrained Annealing-based Sampling MPC (CAS-MPC), which integrates inequality constraints via a primal–dual weight reshaping scheme that preserves sample diversity, and a hybrid refinement strategy that switches to Sequential Linear–Quadratic MPC for millimeter-level precision once near the goal is proposed.
Chenghao Wang, Diego Romeres, Alexander V. Schperberg et al.· 0 citations
Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for determinis...
Yashdeep Chaudhary, R. Armellin, Harry Holt· 0 citations
A Constrained Reinforcement Learning framework for quadrupedal mobile manipulators that learns a single whole-body policy for tracking a 6D end-effector pose specified in task space, with operational limits enforced during learning through a constrained learning mechanism.
Álvaro Belmonte-Baeza, M. Cazorla, G. García et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026