A framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits.
Abstract
Physical interactions can create future hazards that are not apparent from the robot's current geometric surroundings. We present a framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering. A vision-language model (VLM) predicts physical events and their timing or directly predicts object displacements. An explicit motion model converts event hypotheses into object trajectories. Split conformal prediction calibrates position errors jointly across specified objects, observation times, and future times; geometric shape bounds convert the resulting position regions into predicted object occupancy. PSS evaluates a prescribed backup maneuver against this occupancy and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits. MuJoCo experiments with a Unitree Go1 consider falling fixtures, impact-driven support loss, and contact propagation. PSS achieves a safe episode rate of 99.3%, compared with 43.3% for a Backup Control Barrier Function baseline that only uses current obstacle geometry.
Navigation for humanoid robots is critical, yet large-scale evaluation on physical hardware is often impractical due to cost and safety concerns, making simulation benchmarks essential. Existing VLN benchmarks achieve physically executable navigation, but still assume (1) all hazards are observable from elevation maps;...
Shun-Yu Yao, Song-Yan Liu, Ding-Hao Chen et al.· 0 citations
Safe navigation in human crowds requires efficiency and reliability. Learning-based policies run fast but can fail under distribution shift, whereas Hamilton–Jacobi (HJ) reachability offers principled safety guarantees but is computationally expensive. We propose a prediction-conditioned reachability framework that bri...
I. Winata, Junghyun Oh· IEEE Robotics and Automation...· 0 citations
Vision-language-conditioned robot policies integrate perception, language understanding, and control for general-purpose manipulation. However, existing evaluations often focus on task success, isolated physical constraints, semantic refusal, or realized physical damage, providing limited insight into where safety fail...
Jin-Zhu Luo, Qi Zhang, Wei Wang et al.· arXiv.org· 0 citations
This work formalizes the reasoning-execution boundary as a typed contract and constrains language-level decisions through schema-validated tool calls defined by the Model Context Protocol, rejecting malformed commands before they reach the robot.
Autonomous robots operating in partially observed environments must navigate safely while acquiring observations that improve future planning. Existing safety formulations generally reason primarily about geometry. Consequently, geometrically similar scene elements may induce comparable control responses despite having...
Robotic-GST is presented, a geometry-aware spatio-temporal behaviour representation and evaluation framework that constructs a Gaussian-SAM robotic environment for real-to-sim policy verification and improves the reliability of real-world manipulation deployment.
Si-Chao Liu, Ze-Kun Wang, Li-Xuan Tang et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.