Mixture-of-Experts (MoE) large language models (LLMs) are increasingly deployed in Web-facing services, where inference must be both accurate and responsive under bursty demand. Although MoE models improve parameter efficiency through sparse expert activation, efficient MoE inference remains challenging in practice. A major reason is the highly imbalanced expert access pattern during inference: a few hot experts process most routed tokens, while many cold experts are rarely activated, leaving GPU parallelism underutilized. Existing systems mainly optimize runtime execution, such as scheduling, communication overlap, and kernel fusion, but usually preserve the original expert organization and therefore do not address the structural inefficiency caused by fragmented expert usage. In this paper, we present \textbf{BrownoutMoE}, a structure-aware optimization framework for efficient and accurate MoE inference services. Inspired by the brownout paradigm in service computing, BrownoutMoE reorganizes experts into groups to improve utilization and system efficiency while maintaining service quality. Specifically, we formulate layer-wise expert grouping as a learning problem and employ reinforcement learning to discover grouping strategies that minimize accuracy degradation. We further introduce a grouping-consistent distillation process to produce deployable models that are compatible with standard inference pipelines. Experimental results demonstrate that BrownoutMoE reduces accuracy degradation by up to 71.4% and improves throughput by up to 2.24x over baselines.
Yi Ding, Minxian Xu, Zhengxin Fang et al.· 0 citations
While spot instances offer a cost-effective alternative to on-demand cloud resources, they introduce reliability challenges for latency-sensitive microservices due to preemption risks and unpredictable provisioning delays. Conventional resource management systems, which often rely on assumptions of immediate instance availability, fail to account for these operational realities—resulting in increased risk of SLO violations when deployed in spot-based environments. In this paper, we propose Cremes, an adaptive and cost-efficient scaling framework that ensures microservice recovery within the spot instance grace period. Cremes explicitly models both instance waiting time and microservice startup latency, leverages cloud-exposed availability metrics, and applies lightweight machine learning for end-to-end latency prediction. By integrating these components into a multi-dimensional optimization engine, Cremes minimizes cost while satisfying recovery and performance constraints. Evaluations on AWS instances using DeathStarBench, TrainTicket, and Alibaba trace-driven experiments show that Cremes reduces infrastructure cost by up to 37.1% and maintains SLO violation rates under preemptible environments below 6.7%.
Liao Chen, Chenyu Lin, Junlin Chen et al.· IEEE International Symposium...· 0 citations
Driven by the exponential growth of latency-sensitive applications, mobile edge computing (MEC) has emerged as a pivotal paradigm, yet mitigating its substantial energy consumption remains critical. This paper explores a movable-antenna (MA) assisted energy minimization scheme in an uncrewed aerial vehicle (UAV)-enabled MEC system, where a UAV equipped with an MA array serves as an edge server to process tasks offloaded from terrestrial consumer electronics (CE) devices. To minimize the total system energy consumption, we jointly optimize computation resource allocation, CE transmit power, receive beamforming, and MA positions. To tackle the resulting non-convex problem with coupled variables, a robust alternating optimization algorithm based on the block coordinate descent method is developed. The problem is iteratively decomposed into three subproblems. In particular, the subproblem of transmit power and receive beamforming is reformulated and optimized using the quadratic transform technique, while the MA array positions are optimized via the particle swarm optimization algorithm. Numerical simulations verify that the proposed scheme achieves substantial energy savings over conventional benchmarks.
Jiang Chen, Chunjie Wang, Xuhui Zhang et al.· IEEE transactions on consume...· 0 citations
CoScale is proposed, a layer-wise dynamic scaling mechanism that selectively expands the parallelism of hot layers onto idle resources reclaimed from underutilized devices, enabling elastic data parallelism without altering model architectures or adding hardware overhead.
Jingfeng Wu, Yiyuan He, Minxian Xu et al.· 0 citations
Large language model (LLM) deployment at the network edge faces a fundamental paradox: applications require full-scale models for sophisticated reasoning, yet edge devices impose severe resource constraints across computation, memory, and network. Existing approaches fail to effectively orchestrate resources across the edge-cloud continuum, leaving capacity underutilized while struggling with heterogeneous and volatile distributed environments. We present DynoPipe, an adaptive edge-cloud system that addresses these constraints through dynamic pipeline parallelism with shifting computational boundaries. DynoPipe tackles three core challenges: structural heterogeneity causing 94% pipeline idle time, temporal resource volatility invalidating static partitioning, and boundary migration overhead trapping systems in suboptimal configurations. Through boundary-constrained pipeline construction, proactive multi-configuration orchestration, and hierarchical state management, DynoPipe eliminates the memory wall while preserving data locality, achieving $\mathbf{1 0. 1} \times$ throughput improvement over edge-only baselines and $\mathbf{1. 6} \times$ over cloud-only execution, with 99.2% latency reduction.
Yanying Lin, Baicheng Chen, Xinyu Zhang et al.· International Symposium on C...· 0 citations