Generative AI Model Assisted Multimodal Semantic Vehicular Edge Computing for Collaborative Perception

Abstract

Vehicular edge computing (VEC) plays a pivotal role in enabling cooperative perception for connected autonomous vehicles (CAVs), providing comprehensive environmental awareness for safe vehicle control and road safety. However, collaborative perception in VEC faces significant challenges arising from stringent bandwidth constraints, communication latency, and high computational demands. In this paper, we propose a novel multimodal semantic vehicular edge computing (SVEC) architecture for cooperative perception to address these challenges. In the proposed SVEC framework, multimodal sensor data is intelligently selected for task-oriented sharing, and data modalities are adaptively transformed in response to dynamic communication and computing resource constraints. The selected information is then transmitted using advanced semantic encoding and decoding techniques, significantly reducing bandwidth consumption. To compensate for potential information loss introduced by semantic compression, we design a generative AI–based enhancement mechanism to preserve high perception fidelity. We further formulate a joint optimization problem to maximize the vehicles’ quality of experience (QoE) for cooperative perception. To solve this problem, we develop a diffusion-based multi-agent reinforcement learning algorithm that improves exploration efficiency and jointly optimizes vehicle policies in high-dimensional and complex state spaces. Extensive simulation results show that the proposed scheme significantly outperforms the benchmarks, reducing the average system latency by about 28%, decreasing the average system cost by about 19%, and lowering the semantic perception error by up to 40%, while achieving the highest average rewards and average QoE.