The multi-agent transformer (MAT) is adopted to model inter-queue dependencies via attention over agents' observations and actions, enabling implicit coordination across heterogeneous co-located XR applications and results show that the proposed method outperforms baselines.
Abstract
Time-Sensitive Networking (TSN) and Mobile Edge Computing (MEC) hold strong potential for enabling ultra-reliable low-latency communication for time-sensitive applications, such as eXtended Reality (XR). However, the widespread adoption of XR introduces significant challenges due to co-located services in MEC environments, leading to contention for shared network resources. Moreover, XR traffic types have distinct characteristics and criticality in terms of timing requirements, further increasing the complexity and dynamics of such environments. Although reinforcement learning has shown promise for TSN scheduling optimization in dynamic network scenarios, existing approaches rely on centralized or high-level multi-agent designs and are typically tailored to periodic and predictable industrial traffic, limiting their applicability to XR workloads. As a result, these approaches suffer from (i) limited ability to capture inter-queue dependencies due to coarse-grained control, and (ii) poor adaptability to highly dynamic and heterogeneous XR traffic. To address these gaps, we propose a multi-agent reinforcement learning approach for queue-level XR traffic scheduling. We adopt the multi-agent transformer (MAT) to model inter-queue dependencies via attention over agents'observations and actions, enabling implicit coordination across heterogeneous co-located XR applications. Our simulation results show that the proposed method outperforms baselines, achieving up to 71.42% latency reduction and up to 83.2% reduction in failure rate, while consistently achieving high reliability across all queues.
This paper proposes a multi-agent reinforcement learning (MARL) framework for TSN scheduling, where each TSN queue is modeled as an autonomous agent and the Heterogeneous-Agent Proximal Policy Optimization (HAPPO) algorithm is employed to explicitly model inter-agent dependencies and jointly optimize service delivery across queues.
Marcos Carvalho, Fatih Temiz, Shavbo Salehi et al.· 0 citations
High-velocity workloads and intricate task dependencies inherent in distributed stream-processing systems pose a fundamental challenge to efficient resource allocation. Traditional heuristic and single-agent reinforcement learning (RL) schedulers frequently fail to recognize these complex network and data-flow interactions, leading to severe resource fragmentation and catastrophic tail latency spikes. In order to accomplish coordinated, low-latency scheduling, we propose a Topology-Aware Multi-Agent Reinforcement Learning (TAMARL) framework utilizing a Centralized Training and Decentralized Execution (CTDE) architecture. TAMARL allows distributed agents to optimize task placement across heterogeneous cluster nodes and prevent backpressure cascades by integrating topology-aware state representations. We evaluate TAMARL on a production-grade cloud-native stack leveraging Apache Flink and Kubernetes. Compared to state-of-the-art baselines across six demanding stress-test scenarios, experimental evaluations demonstrate that TAMARL improves Service Level Objective (SLO) attainment by 27% while reducing P99 tail latency by up to 68%. Additionally, TAMARL maintains stable, resilient performance under 90% cluster utilization while securing 95% network locality.
Sunday J. Awine, Jinwei Liu· IEEE International Conferenc...· 0 citations
3GPP Access Traffic Steering, Switching, and Splitting (ATSSS) enables traffic to be distributed across heterogeneous 3GPP and non-3GPP access networks to improve performance, reliability, and resilience. ATSSS can use multipath transport protocols such as Multipath QUIC (MP-QUIC), where packet scheduling plays a central role in determining latency and resource utilization for delay-sensitive applications. Many existing MP-QUIC scheduling policies rely on instantaneous path measurements or fixed rules rather than forecasts of future application demand. In multi-flow scenarios, such decisions can lead either to contention on the preferred low-latency path and transient latency inflation for priority traffic or to overly conservative use of available capacity. This paper proposes iScavenger, a predictive, machine-learning-based multi-flow scheduling policy for ATSSS environments. iScavenger employs a Long Short-Term Memory (LSTM) model to forecast near-future bandwidth demand for delay-sensitive Sticky traffic. Based on this prediction, background packets are admitted to the preferred low-latency path only when sufficient residual capacity is expected to remain; otherwise, they are steered to the alternative path. The policy is implemented within the Monty MP-QUIC framework and evaluated in a controlled Mininet testbed using traffic traces from the online game League of Legends, with fixed and variable path capacities and controlled jitter and packet loss. The results indicate that, under the evaluated conditions, iScavenger provides configurable operating points in the latency–utilization trade-off, limiting additional Sticky-flow RTT while achieving higher background throughput than conservative baseline policies. These findings highlight the potential of short-term traffic-demand prediction for proactive contention management in ATSSS-enabled multi-access networks.
Shah M. Emad Uddin, Karl-Johan Grinnemo, Arunselvan Ramaswamy et al.· IEEE Open Journal of the Com...· 0 citations
The lack of determinism restricts the integration of safety-critical applications into Edge–Fog–Cloud (EFC) architectures. Existing EFC schedulers are typically designed for dynamic, best-effort operation based on unmanaged resource allocation and elastic virtualization. This paradigm introduces unbounded queueing, resource contention, and timing jitter, making standard schedulers unsuitable for hard-deadline workloads. Moreover, most approaches focus on computational placement, while communication is abstracted or treated as a secondary cost term. As a result, bounded-latency routing and deterministic task execution are rarely co-optimized under a unified timing model. This paper addresses these gaps by utilizing a managed Time-Triggered Edge–Fog–Cloud (TTEFC) architecture that supports safety-critical workloads, orchestrates IEEE Time-Sensitive Networking (TSN) for local intra-domain communication, and uses IETF Deterministic Networking (DetNet) for routed inter-domain paths. On this infrastructure, a hierarchical genetic algorithm (HGA) is proposed to jointly schedule partition-to-execution-location allocation, partition execution order, inter-partition route selection, and negotiated per-partition time budgets that act as temporal boundaries for parallel partition-level optimizers. An adaptive slack reallocation operator redistributes unused temporal slack from over-satisfied partitions to budget-violating partitions, improving feasibility convergence. Experiments on synthetic DAG workloads with 100–500 tasks compare the proposed HGA against HEFT and round-robin baselines. These baselines are included as scoped external references to contextualize the end-to-end scheduling performance of the proposed method. Ablation results show that slack reallocation improves partition-budget feasibility, reaches feasible budget assignments earlier, and produces tighter budget–makespan alignment than feedback-free and static-budget variants. An automotive-characteristic DAG case study further evaluates the method on an application-oriented workload under the same timing and communication assumptions.
Omar Hekal, Josepaul Paulachan, Daniel Onwuchekwa et al.· Future Internet· 0 citations
The rapid proliferation of Internet of Things (IoT) devices has intensified demands for low-latency, resource-efficient task scheduling at the network edge. Conventional policies such as Round-Robin and First-Come-First-Serve (FCFS) fail to satisfy the Quality-of-Service (QoS) requirements of Industrial-IoT and autonomous-vehicle workloads. This paper presents Multi-Queue Priority-Based Scheduling (MQPBS), a lightweight algorithm that classifies tasks into three dynamic priority queues (High, Medium, Low) using deadline-aware heuristics, applies Shortest-Job-First (SJF) intra-queue ordering, and employs an aging mechanism to prevent starvation. Extensive simulation over task sets of 200–1000 tasks demonstrates that MQPBS reduces average waiting time by up to 17.6%, improves throughput by up to 10.8%, lowers energy consumption by 20%, and cuts the Deadline Miss Ratio (DMR) compared with the Priority-Aware Task-Scheduling (PaTS) baseline. Ablation experiments confirm the independent contribution of each algorithmic component. Scalability and sensitivity analyses further validate the robustness of MQPBS under heterogeneous arrival patterns and varying load intensities. The results establish MQPBS as a scalable, reliable scheduler for next-generation edge infrastructures.
Shibang Maity, Roshan Panda, M. Tanisha et al.· International Conference on...· 0 citations
A hybrid reinforcement learning (RL) framework that jointly controls queue management and bandwidth allocation in bursty multi-service networks and demonstrates the effectiveness of coordinated learning-based control for stable and QoS-aware operation in bursty networked systems.
T. Khan, Babar Shah, Taimur Karamat et al.· Computing· 0 citations
