HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference

CHIME: Cost-Constrained Hybrid Popularity-Aware Intelligent Service Caching Framework for MEC

Mobile edge computing (MEC) has emerged as a promising paradigm to support latency-sensitive applications by deploying computing and storage resources closer to end users. Service caching at the network edge is an effective approach to further reduce response latency and improve quality of service (QoS). However, the increasing scale and dynamic nature of user requests pose significant challenges to efficient service caching and updating under limited edge resources and cost budgets. In this article, we investigate the joint optimization problem of service caching and dynamic updating in a cloud–edge–end collaborative architecture, with the objective of minimizing the average service access latency of users. We first construct a comprehensive system model that captures communication latency, service response delay, and the costs associated with service caching and updates. The problem is formulated as a nonlinear integer programming problem and addressed via a two-stage solution framework. Specifically, in the initial caching stage, we employ Lagrangian relaxation to reduce the high-dimensional knapsack problem and design an efficient algorithm integrated with Tabu Search. In the dynamic updating stage, we develop a Monte Carlo tree search (MCTS)-based algorithm enhanced by service popularity awareness, along with a novel service replacement strategy that considers user preference and service reliability. Extensive experiments under various system scales demonstrate that the proposed approach significantly outperforms state-of-the-art methods in terms of average service latency, cost budget satisfaction, and update efficiency.

HAT-Route: A Physics-Aware Hierarchical Transformer Framework for Scalable Cloud-Edge Collaborative Routing

Routing optimization in cloud-edge collaborative networks faces a fundamental conflict between global strategic planning and local real-time responsiveness, further complicated by structural heterogeneity and stochastic traffic patterns. Traditional protocols lack adaptivity, while existing Deep Reinforcement Learning (DRL) approaches based on Graph Neural Networks (GNN) struggle with limited receptive fields and over-smoothing issues in large-scale topologies. In this paper, we propose HAT-Route, a Transformer-driven hierarchical routing framework supported by the Network Digital Twin (NDT). Our contributions are threefold: 1) We establish a cloud-edge collaborative architecture operating under the Centralized Training and Decentralized Execution paradigm. This architecture balances the trade-off between global optimization and real-time inference. 2) We introduce FlowFormer, a Spatiotemporal Transformer for the NDT. FlowFormer integrates a novel Edge-Conditioned Spatial Attention (EC-SAT) mechanism to capture physical link constraints and distinguish between congestion and Head-of-Line (HOL) blocking. 3) We design HAT-Route, a hierarchical DRL agent that utilizes Graph Transformers for global policy learning in the cloud, coupled with knowledge distillation to deploy lightweight policies at the network edge. Extensive experiments demonstrate that our framework outperforms traditional protocols and GNN-based baselines in terms of QoS optimization, training stability, scalability, and generalization capability on large-scale network topologies.

A Cooperative MADRL Approach for Joint Computing Offloading and Route Optimization in Cloud-Edge Networks

Future 6G networks will integrate communication and computing capabilities to support intelligent, delay-sensitive services. In heterogeneous cloud-edge environments, however, task offloading and routing decisions are strongly coupled, and dynamic workloads, limited computing resources, and constrained link capacity make efficient service provisioning challenging. Existing reinforcement learning-based offloading methods can improve decision efficiency, but many focus on simplified or single-domain settings and do not adequately account for backbone topology and bandwidth constraints. To address this problem, this paper studies joint task offloading and routing optimization in multi-domain cloud-edge networks, explicitly modeling network topology and link capacity. We propose a cooperative multi-agent deep reinforcement learning method that coordinates distributed edge agents through centralized training and decentralized execution. Routing optimization feedback is further incorporated to guide constraint-aware policy learning. Simulation results demonstrate that the proposed method reduces end-to-end latency, mitigates network congestion, and avoids link and node overload in cloud-edge networks.

OrderMoE: An expert similarity driven distributed edge MoE inference

Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it remains challenging to deploy MoE inference over resource-constrained and bandwidth-limited edge infrastructures. Existing distributed MoE serving methods mainly rely on exact expert placement, caching, replication, or communication scheduling, while overlooking the functional similarity among experts, which provides an opportunity to reduce cross-server token transmission. Therefore, this paper introduces a similarity-aware expert allocation and distributed deployment framework, dubbed OrderMoE, which aims to accelerate edge MoE inference while balancing inference latency, communication overhead, server workload, and inference quality. OrderMoE first constructs an expert similarity model based on router-induced logits representations and partitions experts in each MoE layer into multiple similarity groups. Then, it develops a similarity-aware expert grouping and deployment strategy to improve local similarity coverage across edge servers. Since reducing remote expert invocation and preserving exact inference quality are conflicting objectives, OrderMoE further designs a quality-aware and trajectory-aware runtime server-expert selection algorithm to decide whether a token should invoke its remote target expert or use a feasible local substitute expert. Experimental results on a real distributed edge testbed show that OrderMoE significantly reduces average latency, tail latency, cross-server traffic, and remote expert invocation ratio, while introducing only small and controllable inference quality degradation.

TrimMoE A communication aware and adaptive depth framework for distributed edge inference

Serving Mixture-of-Experts (MoE) large language models across distributed edge servers is bottlenecked by the cross-server expert transmission. The existing approaches mainly focus on how to reach a remote expert faster. However, in this paper, we instead consider whether a given layer, and the layers after it, need to be executed at all. To this end, a communication-aware adaptive-depth framework is proposed in this paper, termed TrimMoE, which couples layer skipping and confidence-based early exit with substitute execution and server-expert selection under a unified quality budget. Specifically, in the offline stage, TrimMoE freezes the backbone, trains the lightweight per-layer exit heads, calibrates the per-layer importance thresholds, and allocates the expert replicas by a skip/exit-aware redundancy benefit. In the online stage, a transition-aware look-ahead anticipates the token movement, so that the depth reduction targets the costliest transmissions, and besides, two feedback rules adapt the delay-quality weights and the exit threshold. Moreover, we prove that the substitution-and-skipping proxy degradation never exceeds the configured budget, and that the early exit is admitted only under a calibrated confidence gate. On a heterogeneous 10-server testbed with Switch-Base-8E, Qwen-MoE-A2.7B, and Mixtral-8x7B, TrimMoE reduces the average latency by up to 62.8%, lowers the cross-server traffic and the remote-execution ratio, and sustains high throughput under load, while keeping the task-quality degradation within a 2% bound.

Sparsity-Aware Scheduling for Efficient MoE-Based LLM Serving on Edge Servers