Diffusion-Enhanced Flow Policy Optimization for Cooperative RSMA-HARQ in Integrated Satellite–Terrestrial Networks
Abstract
This paper introduces a cooperative non-orthogonal transmission protocol that integrates rate-splitting multiple access (RSMA) and hybrid automatic repeat request (HARQ) to provide reliability- and timeliness-oriented services for integrated satellite-terrestrial networks (ISTNs). Specifically, a base station is employed to cooperate with a low-Earth-orbit (LEO) satellite to retransmit information within the same satellite transmission round, thereby enabling the system to achieve dual-path diversity gain. We derive analytical expressions for the outage probability (OP) and the average age of information (AoI) of the proposed RSMA-HARQ protocol, which serve as metrics for reliability and timeliness, respectively. These expressions provide insights into how transmit power and rate affect the reliability and timeliness of the protocol. Building upon these insights, we formulate a long-term statistical energy efficiency maximization problem taking into account the OP and AoI requirements. To solve this problem, we propose a diffusion model-enhanced flow policy optimization (DEFPO) algorithm for joint power and rate allocation across the cooperative transmission. By employing the diffusion model as a policy network in deep reinforcement learning (DRL), high-dimensional actions are generated through a reverse process, and an advantage-weighted conditional flow matching loss is utilized to better capture multimodal action patterns. Simulation results demonstrate that the proposed protocol and algorithm outperform existing benchmarks in terms of reliability, timeliness, and energy efficiency. For example, our approach improves energy efficiency by 22.5% under the same reliability and timeliness constraints compared to other DRL-based methods.