Sep 2026· Proceedings of the International Conference on Parallel Processing· 0 citations· 12 references
TL;DR
A Multi-State Semantic Cache (MSSC) framework that dynamically maps knowledge chunks into three mutually exclusive states: zero cache, index cache, and full cache is proposed that reduces average service latency by over 40% in dynamic scenarios and suppresses backhaul reconfiguration traffic by up to 38% compared to conventional binary caching baselines.
Abstract
Enterprise applications are increasingly adopting collaborative Retrieval Augmented Generation (RAG) frameworks that synergize Small Language Models (SLMs) at the network edge with Large Language Models (LLMs) in the cloud for joint reasoning. However, this paradigm is bottlenecked by the capacity asymmetry between massive cloud-based knowledge repositories and resource-constrained edge servers. Existing edge-cloud collaborative frameworks rely on monolithic storage strategies that fail to exploit the structural asymmetry between the vector index of knowledge chunks and raw documents. This limitation induces a severe conflict between service latency and cache reconfiguration traffic. This paper proposes a Multi-State Semantic Cache (MSSC) framework that dynamically maps knowledge chunks into three mutually exclusive states: zero cache, index cache, and full cache. By leveraging semantic graph diffusion, our framework captures conceptual dependencies to predict query trajectories, enabling predictive state transitions among three states. Furthermore, we formulate the cache scheduling process as a Model Predictive Control (MPC) problem to optimize the trade-off between service latency and reconfiguration traffic. Experimental evaluations on real-world datasets demonstrate that, while strictly preserving generation quality, our framework reduces average service latency by over 40% in dynamic scenarios and suppresses backhaul reconfiguration traffic by up to 38% compared to conventional binary caching baselines.
AceSpec, an asymmetric edge-cloud collaborative framework that employs an asymmetric communication protocol that transmits minimal main-chain indices uplink and compact sparse distributions downlink and introduces a network-aware, Lagrangian-optimized resource allocation strategy that dynamically maximizes the local ca...
Yi-Da Zhang, Zhi-Yong Gao, Shuai-Bing Yue et al.· 0 citations
Pervasive intelligent applications are increasingly deployed on mobile and Internet of Things (IoT) edge devices. Consequently, Large Language Models (LLMs) are increasingly used to support these applications. Yet, due to their high resource demands, LLMs are mostly deployed in the cloud. Layer-wise edge-cloud inferenc...
Jing-Po Xu, Paul Joe Maliakel, Ivona Brandić et al.· 0 citations
Deploying Large Language Models (LLMs) over the edge-cloud continuum faces severe stability challenges due to the conflict between stochastic network topology and complex workflow dependencies. Existing schedulers, relying either on computationally prohibitive Graph Neural Networks (GNNs) or topology-agnostic heuristic...
Yan Gao, Shaoyuan Huang, Yonghui Ye et al.· IEEE Transactions on Cogniti...· 0 citations
This paper proposes a spatiotemporal hypergraph-driven framework integrating high-order topological feature extraction with dynamic resource modeling, and introduces dynamic hypergraph sequences to naturally encompass local conflict domains, mitigating the topological blind spots and “over-smoothing” issues inherent in...
Kun Ding, Xi-Wen Qiu, Nian-Feng Weng et al.· Journal of Cloud Computing· 0 citations
This work introduces, for the first time, a personalized bilevel optimization framework that formalizes cloud-edge LLM collaboration as a dual structure: the upper level optimizes edge-side personalization, while the lower level governs cloud-side knowledge transfer, reaching cloud-edge evolving in coordination.
Victor Chen, Hai-Rui Yu, S. K. Chung et al.· arXiv.org· 0 citations
Deploying large language models (LLMs) at the network edge requires low-latency inference despite limited GPU memory and frequent model switching. Conventional caching strategies are ineffective because semantically similar queries may differ significantly in surface form. To address this issue, we propose a unified fr...
Sai Wang, Chang-Sheng You, Xiao-Yang Li et al.· International Symposium on W...· 0 citations