Skip to content

AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference

Sep 2026 · 0 citations · 25 references
Computer Science

TL;DR

AceSpec, an asymmetric edge-cloud collaborative framework that employs an asymmetric communication protocol that transmits minimal main-chain indices uplink and compact sparse distributions downlink and introduces a network-aware, Lagrangian-optimized resource allocation strategy that dynamically maximizes the local cache hit rate.

Abstract

Deploying Large Language Models (LLMs) on edge devices typically relies on model compression or split inference. However, compression degrades reasoning capabilities, while split inference suffers from severe Wide Area Network (WAN) communication bottlenecks. Edge-cloud speculative decoding emerges as a promising alternative, leveraging an edge small model to draft tokens for cloud verification. Yet, over volatile WANs, inevitable prediction rejections trigger catastrophic pipeline stalls and network-wide rollbacks, neutralizing collaborative gains. To overcome this, we propose AceSpec, an asymmetric edge-cloud collaborative framework. AceSpec utilizes un-saturated edge compute to proactively construct a probabilistic state cache, effectively transforming network-wide pipeline flushes into $\mathcal{O}(1)$ local memory lookups. To preserve bandwidth, it employs an asymmetric communication protocol that transmits minimal main-chain indices uplink and compact sparse distributions downlink. Furthermore, we introduce a network-aware, Lagrangian-optimized resource allocation strategy that dynamically maximizes the local cache hit rate. Evaluations demonstrate that AceSpec achieves up to a 3.52$\times$ throughput speedup and exhibits exceptional bandwidth immunity, sustaining near-peak inference performance even under severely constrained 50 Kbps WAN conditions.

View source

Similar papers

#edge computing Open access Dec 2026

Scaling LLMs via Collaboration: Efficient LLM Inference Across Resource-Constrained Devices

CoSEI is presented, a system that introduces operator-aware partitioning and synchronization-minimizing execution, guided by a lightweight planner and a tailored runtime, that demonstrates that with the right design, collaborative edge inference can make LLMs practical beyond the cloud.

Fei Zeng, Feng Lyu, Hao Wu et al. · 0 citations
#artificial intelligence Preprint Oct 2026

DySCo: Dynamic Sharding for Collaborative Edge-Cloud LLM Inference with Depth-Synchronized Batching

Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models (LLMs) are increasingly used to support these applications. Yet, due to their high resource demands, LLMs are mostly deployed in the cloud. Layer-wise edge-cloud inferenc...

Jing-Po Xu, Paul Joe Maliakel, Ivona Brandić et al. · 0 citations
Preprint Oct 2026

DS-Net: A Dynamic Split Network for Edge-Cloud Collaborative Inference

Collaborative inference serves as a critical methodology in edge intelligence. Existing collaborative inference methods operate by splitting the deep neural network (DNN) model between edge devices and cloud servers. However, these methods need to load different models for multiple split points when the communication o...

Xin-Yun Zhang, Li Wang, Lei Jin et al. · 0 citations
Preprint Oct 2026

CEMA-Router: Infrastructure-Aware Batch Routing for LLM Inference in Cloud-Edge Systems

Large language models (LLM) are increasingly integrated into pervasive applications spanning heterogeneous cloud-edge resources. These heterogeneous resources differ in computational capacity, energy efficiency, network conditions, and deployed model quality, creating trade-offs among response quality, monetary cost, e...

Xi-Jia Wang, Tian-Zhang He, Mohammad Goudarzi · 0 citations
#edge computing Book Open access Oct 2026

EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures

This work presents EdgeAgent, a cross-layer inference system explicitly co-designed for edge UMA and multi-agent workloads, and demonstrates that the UMA-aware execution alone contributes a 1.29x speedup over batched speculative decoding and adding the agent-aware scheduling lifts the full EdgeAgent system to a 1.77x s...

Yu-Hai Long, Yuan-Xin Wei, Kai Wu et al. · 1 citation
#large language models Book Open access Dec 2026

CoordSpec: Coordinating Edge Drafting and Cloud Verification for Efficient Speculative Decoding

This work presents CoordSpec, a framework that coordinates cloud verification scheduling with adaptive draft length selection, and presents the system design and an evaluation plan under varying task loads and network conditions.

Dongkyun Ryoo, Jihoon Lim, Daesik Kim et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.