Real-time quantized BEV-LLM autonomous driving under varying traffic densities

Abstract

Closed-loop autonomous driving at 10 Hz leaves approximately 100 ms for perception and control inference, making fullprecision 7B large language models unsuitable for deployment. This paper evaluates whether GPTQ 4-bit quantization and paged inference can reduce decision latency enough for BEV-to-text driving control as traffic density and prompt length increase. A single top-down camera is processed by a bird's-eye-view (BEV) perception front end that extracts vehicle locations, headings, and lane assignments; a rule-based encoder then converts the scene into textual prompts for a LoRAadapted Qwen2.5-7B decision model. Experiments in CARLA 0.9.14 show that quantization reduces resident GPU memory from 14.2 GB to 5.8 GB. Mean decision latency remains within 81–97 ms in sparse and moderate traffic and rises to 110–113 ms in dense traffic. The largest improvement appears in moderate traffic, where latency decreases from 362– 412 ms to 96–97 ms relative to the non-accelerated model. Success rate declines from 94 % in sparse traffic to 83–84 % in dense traffic, yet the quantized method still outperforms TransFuser across all tested densities. These results indicate that GPTQ 4-bit quantization and paged inference make BEV-LLM decision-making practical for near-real-time closedloop driving in controlled traffic, while dense traffic remains constrained by prompt growth and lane-change precision.