FailBench: How Reliable are VLMs at Judging Robot Task Success?
This work introduces FailBench, a benchmark for robot failure detection comprising 2,197 manipulation attempts across 14 public sources (12 real-world, 2 simulated), and shows that input-level intervention--spatially localizing and cropping outcome-relevant regions--improves the top detector by 2.4 percentage points wi...