Skip to content

Predictive Semantic Safety: From Visual Physical Reasoning to Safety-Critical Control

Sep 2026 · arXiv.org · Vol abs/2609.34356 · 1 citation · 23 references
Computer Science

TL;DR

A framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits.

Abstract

Physical interactions can create future hazards that are not apparent from the robot's current geometric surroundings. We present a framework termed Predictive Semantic Safety (PSS), which connects visual physical reasoning to backup-based safety filtering. A vision-language model (VLM) predicts physical events and their timing or directly predicts object displacements. An explicit motion model converts event hypotheses into object trajectories. Split conformal prediction calibrates position errors jointly across specified objects, observation times, and future times; geometric shape bounds convert the resulting position regions into predicted object occupancy. PSS evaluates a prescribed backup maneuver against this occupancy and derives input-affine constraints for minimally modifying the nominal input while preserving backup feasibility under the robot dynamics and input limits. MuJoCo experiments with a Unitree Go1 consider falling fixtures, impact-driven support loss, and contact propagation. PSS achieves a safe episode rate of 99.3%, compared with 43.3% for a Backup Control Barrier Function baseline that only uses current obstacle geometry.

View source

Similar papers

Preprint Oct 2026

What the Elevation Map Cannot See: Semantic-Aware Locomotion and Execution-Aware Navigation for Humanoid Robot

Navigation for humanoid robots is critical, yet large-scale evaluation on physical hardware is often impractical due to cost and safety concerns, making simulation benchmarks essential. Existing VLN benchmarks achieve physically executable navigation, but still assume (1) all hazards are observable from elevation maps;...

Shun-Yu Yao, Song-Yan Liu, Ding-Hao Chen et al. · 0 citations
Open access Nov 2026

Uncertainty-Calibrated Hamilton-Jacobi Value Learning for Safe Crowd Navigation

Safe navigation in human crowds requires efficiency and reliability. Learning-based policies run fast but can fail under distribution shift, whereas Hamilton–Jacobi (HJ) reachability offers principled safety guarantees but is computationally expensive. We propose a prediction-conditioned reachability framework that bri...

I. Winata, Junghyun Oh · 0 citations
Sep 2026

SafeStage: Evaluating Safety Before, During, and After Vision-Language-Conditioned Robot Manipulation

Vision-language-conditioned robot policies integrate perception, language understanding, and control for general-purpose manipulation. However, existing evaluations often focus on task success, isolated physical constraints, semantic refusal, or realized physical damage, providing limited insight into where safety fail...

Jin-Zhu Luo, Qi Zhang, Wei Wang et al. · 0 citations
Preprint Sep 2026

SemSafe-3DGS: Semantic Risk-Aware Active Navigation in Uncertain 3D Gaussian Splatting Maps

Autonomous robots operating in partially observed environments must navigate safely while acquiring observations that improve future planning. Existing safety formulations generally reason primarily about geometry. Consequently, geometrically similar scene elements may induce comparable control responses despite having...

Amirhossein Mollaei Khass, Athanasios Cosse, Nader Motee · 1 citation
#artificial intelligence Preprint Sep 2026

Robot-GST: geometry-aware spatial-temporal robot policy representation and evaluation

Robotic-GST is presented, a geometry-aware spatio-temporal behaviour representation and evaluation framework that constructs a Gaussian-SAM robotic environment for real-to-sim policy verification and improves the reliability of real-world manipulation deployment.

Si-Chao Liu, Ze-Kun Wang, Li-Xuan Tang et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.