LASA: Latency-Aware Safety Arbitration for Vision-Language-Action Autonomous Driving

Abstract

Deploying Vision-Language-Action (VLA) models in real-time autonomous driving faces two barriers: the difficulty of safety assurance for black-box end-to-end outputs, and inference tail latency far exceeding the 20 ms control deadline at 50 Hz, which causes stale decisions to produce dangerous actions. The 50 Hz Fast Reflex layer performs lane detection via INT8-quantized YOLOv8-seg and drives the vehicle using a spline-based path planner. The 1-2 Hz Slow Brain (Qwen3-VL-8B-Instruct, LoRA fine-tuned) runs asynchronously and provides semantic decisions to assist driving in long-tail edge cases. The VLA emits only a Bounded Intermediate Representation comprising a lateral offset, speed scale, validity window, confidence score, and an observation frame timestamp. LASA (Latency-Aware Safety Arbitration) arbitrates this representation at 50 Hz, evaluating staleness against the observation timestamp rather than the IR arrival time, together with asymmetric confidence gating and slew-rate limiting. We validate LASA through a decision-level gate ablation and timing measurements on a physical 1/5-scale platform. Per-gate ablation on a synthetic event stream shows that observation-time arbitration contracts the published command envelope under lane ambiguity - lateral-command RMS drops from 23.2 to 5.4 px and mean speed scale from 0.68 to 0.39 -yielding a markedly smoother and more conservative trajectory than an ungated VLA. On hardware, the off-path distractor false-positive stop rate is 0%, and the 50 Hz reflex loop sustains a sub-millisecond serial-write P99 (0.77 ms) with zero clockmonotonicity violations across the pipeline.