Simulation results show that the proposed DH-MDRL framework outperforms conventional schemes without IRSs and achieves an excellent trade-off between V2V link constraints’ satisfaction probability and V2I link sum data rates compared to centralized resource allocation approaches.
Abstract
Vehicle-to-Everything (V2X) communication is an essential building block of intelligent transportation systems, supporting high-data-rate vehicle-to-infrastructure (V2I) services, and ultra-reliable low-latency vehicle-to-vehicle (V2V) communication. However, in dense urban environments, V2X services can be significantly degraded by the presence of severe blockage, fast channel variations, and high levels of interference. Intelligent Reflecting Surfaces (IRSs) can be employed to reconfigure wireless propagation environments to improve V2X communication. However, the joint optimization of transmit power, spectrum reuse, and IRS reflection coefficients is a mixed-integer non-linear problem, which is further complicated by the fast vehicular mobility and time-varying interference in V2X networks. To tackle this challenging problem, this work proposes a scalable and deployable decentralized hierarchical multi-agent deep reinforcement learning (DH-MDRL) framework. The key design principle is the separation of control timescales, whereby each V2V link functions as an autonomous agent that responds to local observations at a fast timescale and determines its transmit power and spectrum reuse decisions, while the IRS controller at the base station (BS), using global network observations, updates the IRS reflection coefficients at a slower timescale. This hierarchical architecture reduces coordination signaling associated with centralized resource allocation while enabling distributed resource allocation. The IRS-assisted V2X network is modeled as a Markov decision process, where the reward design is tailored to optimize the V2I sum data rate while guaranteeing the latency and reliability constraints associated with safety-critical V2V communication. Simulation results show that the proposed DH-MDRL framework outperforms conventional schemes without IRSs and achieves an excellent trade-off between V2V link constraints’ satisfaction probability and V2I link sum data rates compared to centralized resource allocation approaches.
Future applications in the 6G-based Internet of Vehicles will leverage sidelink (SL) transmissions in Vehicle-to-Everything (V2X) scenarios. However, SL-based direct communication can significantly increase interference among vehicles and between vehicles and other entities of the Intelligent Transportation System. Thus, both Vehicle-to-Vehicle communications and Vulnerable Road Users (VRUs) uplink resources may be degraded or subject to starvation. Existing solutions primarily focus on improving resource allocation and pair selection. Nonetheless, they lack a comprehensive approach to tackle the communication modes and the entire network. To address these challenges, this paper leverages Open RAN to manage V2X communication and proposes a multi-agent reinforcement learning (MARL) resource-aware system. Open RAN provides control loops through a global view of the network and also an open interface-based framework for machine learning models applied to resource decision-making. Meanwhile, the MARL model aims to mitigate interference, optimize resource usage, and enhance quality of service by optimally selecting between sidelink and network transmissions. To reduce system complexity, this work employs a clustering strategy. Each agent manages a group of pairs, rather than assigning one agent to each pair. The solution supports this design by adopting a centralized training with decentralized execution approach, empowered by Open RAN. The strategy uses offline training and an off-policy approach, in which each agent stores experience for fine-tuning. Results indicate that the MARL approach reduces average loss by 21% and latency by 19% in Vehicle-only scenarios. In coexistence VRU scenarios, loss and latency drop by 18% and 30%, respectively, compared to the single-agent approach.
M. Barbosa, K. Dias· IEEE Transactions on Vehicul...· 0 citations
Vehicular edge computing (VEC), a key enabler for the Internet of Things (IoT) in intelligent transportation, addresses onboard processing constraints through collaborative task offloading among vehicles, facilitating latency-sensitive applications such as autonomous driving. However, developing efficient offloading strategies remains particularly challenging in high-density vehicular networks, where intensive computational demands coexist with severely constrained intervehicle communication ranges due to signal blockage. To handle this, we propose M4O, a mobility-aware task offloading framework supporting multihop, multiuser, and multitask offloading optimization. M4O intelligently integrates vehicle mobility patterns and enables relay-assisted offloading to enhance system effectiveness and robustness. The framework employs a dual-algorithm approach: the advantage actor–critic (A2C) for indivisible tasks and the hybrid proximal policy optimization (H-PPO) for divisible tasks, both optimized to minimize the temporally coupled composite cost of time and resources. Extensive experiments demonstrate that the deep reinforcement learning (DRL)-based solutions of M4O deliver stable and efficient offloading strategies, outperforming existing benchmarks by significant margins in cost efficiency. Our code is available at https://github.com/Zhouym1028/M4O
Momiao Zhou, Yimin Zhou, Yanshi Sun et al.· IEEE Internet of Things Jour...· 0 citations
Ensuring reliable and low-latency vehicle-to-everything (V2X) communications in high-speed transport settings remains a significant challenge due to severe path loss brought about by non-line-of-sight (NLoS) and coverage gaps in conventional cellular infrastructure. While dielectric waveguide-based pinching antenna (PA) systems have been proposed to mitigate these physical limitations, they suffer from substantial in-waveguide attenuation over long distances. To address these challenges, we propose a segmented waveguide-enabled pinching-antenna (SWAN) architecture in platoon-based V2X networks. By employing dynamic segment selection, SWAN maintains robust line-of-sight (LoS) connectivity while mitigating the in-waveguide attenuation inherent in conventional PA structures. We formulate a joint resource allocation (RA) and mode selection problem to minimise the age of information (AoI) for both uplink platoon monitoring and downlink traffic broadcasting, whilst ensuring the exchange of intra-platoon cooperative awareness messages (CAMs) and minimising power consumption. To solve this high-dimensional problem, we propose a decomposed multi-agent deep deterministic policy gradient (DE-MADDPG) algorithm augmented with twin delayed (TD3) critics by considering each vehicle platoon (VP) as an agent. This approach decouples system-wide coordination from local executions of VPs, enabling efficient learning in dynamic environments. Extensive simulations demonstrate that the proposed framework significantly outperforms standard reinforcement learning (RL) baseline methods, achieving near-optimal uplink and downlink AoI performance, with an average gap of 4.8% to exhaustive search, and near-perfect CAM delivery probability (CDP), which approaches 100%, even under dense traffic conditions.
Yuxiang Zheng, Simon L. Cotton, T. Q. Duong· IEEE Transactions on Cogniti...· 0 citations
Vehicular edge computing (VEC) is viewed as an important component for vehicle networks due to the prevalence of smart vehicles. However, stringent requirements such as ultralow latency and minimal energy consumption should be achieved for the emerging transmission-and-computation-intensive applications due to the limited communication and computing resources. To address these challenges, this article proposes a multiagent priority-based twin-delayed deep deterministic policy gradient (MAPTD3) approach for the joint optimization of task offloading and power allocation in a VEC system at the millimeter-wave (mmWave) band. First, the joint task offloading and power allocation problem is formulated as a nonconvex mixed-integer nonlinear program (MINLP) with a multiplicative objective function subject to delay and power constraints. Second, the proposed approach integrates two key techniques: a dynamic variance noise (DVN) mechanism for adaptive exploration and prioritized experience replay (PER) to enhance sample efficiency. Finally, it is shown in the simulation results that MAPTD3 consistently outperforms existing benchmarks with 96%–99% optimality ratios (ORs) under various load scenarios.
Zhaobin Li, Qiqi Zhang, Ziqiang Zhang et al.· IEEE Internet of Things Jour...· 0 citations
Integrated sensing, communication, and computation (ISCC) provides a critical enabling platform in supporting the diverse services in the Internet of Vehicles (IoV). However, effective heterogeneous IoV service provisioning relies on both communication-centric and beyond-communication performance metrics, making unified resource allocation challenging. Moreover, competition from concurrent services for limited multi-dimensional resources is intensified in dynamic vehicular environments. In this paper, we investigate the resource allocation problem for concurrent communication and target classification services in an ISCC-enabled IoV system. To solve the problem, we first introduce the value of service (VoS) to unify communication rate and classification accuracy into a common measure that captures the degree of heterogeneous service fulfillment. To reduce the complexity of dynamic problem optimization, we propose a digital twin-assisted proximal policy optimization (DTPPO) algorithm, in which the digital twin exploits both current and historical information to generate predictive information, thereby enhancing policy learning in dynamic environments. Furthermore, we develop a large language model (LLM)-enhanced DTPPO (LLM-DTPPO) algorithm, which leverages the contextual understanding and domain knowledge of LLMs to reshape the reward function and improve resource allocation performance under multi-dimensional resource competition. Simulation results based on real-world vehicle mobility traces demonstrate that the proposed algorithms outperform existing benchmark schemes.
Bangzhen Huang, Zhang Liu, Lianfen Huang et al.· IEEE Transactions on Network...· 0 citations
Future 6G services will require strict performance guarantees, especially in terms of delay, end-to-end (e2e) across multiple network domains including packet and radio segments. While deterministic transport and slice-based capacity allocation can improve segment-level performance, ensuring e2e Network Service (NS) performance remains challenging as it requires making decisions Near–Real-Time (Near-RT) on a per-service basis, which does not fit well within the typical centralized control and orchestration hierarchy. Multi-agent systems (MAS), where a number of distributed agents collaborate, has demonstrated its capabilities for such Near-RT control. Agents equipped with Deep Reinforcement Learning (DRL) engines autonomously made traffic routing decisions based on e2e telemetry measurements. In this paper, we extend such MAS solutions for NS traffic routing focused on covering several issues that appear under frequent NS reconfiguration, e.g., caused by end device mobility. In addition, we define a lifecycle for NS operation that includes the initial MAS deployment, model reconfiguration during operation, and NS reconfiguration. The proposed lifecycle requires the definition of DRL training and validation procedures to produce models ready to be deployed with guaranteed performance under certain network conditions. In addition, model selection algorithms are defined for the lifecycle scenarios. In case of NS reconfiguration, a procedure for probe testing the actual network conditions is proposed to improve model selection. Evaluation across a meaningful set of network and traffic scenarios shows that the MAS is able to maintain e2e delay guarantees under all the lifecycle scenarios.
H. Shakespear-Miles, S. Barzegar, M. Ruiz et al.· IEEE Transactions on Network...· 0 citations
