Multi-Critic Reinforcement Learning for Frame- and Tick-Rate Aware Satellite–Ground Integrated Heterogeneous Edge Networks

Abstract

Satellite–ground integrated networks (SAGIN) enable wide-area support for latency-sensitive interactive applications such as UAV teleoperation and remote robotic control. Unlike conventional data services that focus on average latency or throughput, these applications operate in closed-loop feedback cycles in which perception frames and control actions must be synchronized and satisfy strict deadlines. Late-arriving data often become obsolete rather than recoverable. Moreover, practical edge servers execute tasks in discrete scheduling cycles, introducing tick-quantized completion times that directly affect deadline violations. This paper proposes a heterogeneous satellite–ground mobile edge computing framework in which terrestrial base stations (BSs) and satellites (SATs) both provide computing services with distinct capacity characteristics. We develop a server-side queuing and scheduling model that captures continuous frame generation, stochastic control inputs, buffer constraints, and discrete tick-based task completion. Based on this model, we formulate a multi-objective optimization problem that jointly determines user association, transmission power, and bandwidth allocation to minimize deadline violations, dropped tasks, and worst-case delay. To solve the resulting mixed-integer nonlinear program under dynamic satellite topology, we design a multi-critic reinforcement learning (RL) algorithm that decomposes synchronization, latency, and capacity constraints. Simulation results demonstrate substantial reductions in deadline violations and maximum delay compared with existing approaches, including proximal policy optimization (PPO) deep deterministic policy gradient (DDPG), greedy algorithm, proportional fairness scheme, random access strategy, and equal resource allocation method.