Skip to content

A Cloud--Edge Collaborative Large Language Model Inference Framework Based on Historical Context Matching

Oct 2026 · IEEE Internet of Things Journal · Vol 13, pp. 45369-45385 · 0 citations · 50 references

Abstract

Cloud–edge collaborative inference has emerged as a promising paradigm to address the latency, energy, and privacy challenges of large language models (LLMs). However, current offloading mechanisms often struggle to efficiently capture the dynamic semantic dependencies between historical context and ongoing queries. This oversight results in redundant token processing or imprecise cloud routing, leading to semantic disconnection and suboptimal system utility. To overcome these limitations, we propose a cloud–edge collaborative LLM inference framework with historical context matching (HCM-CELLM), which integrates context awareness with edge computing. Deployed at the edge, HCM-CELLM employs an approximate nearest neighbor (ANN) search and a lightweight Transformer-based attention module to assess semantic relevance, filtering ambiguity and injecting only high-value historical segments. The dynamic matching degree serves as a core state feature for an improved proximal policy optimization (PPO) scheduler, enhanced with a trust-region caching mechanism to enable precision-aware offloading decisions—intercepting highly matched queries at the edge or coordinating cloud-sketch parallel refinement. Extensive simulations demonstrate that HCM-CELLM significantly outperforms state-of-the-art collaborative baselines, achieving superior inference accuracy, task completion rates, and a reduction in latency and energy consumption.

View source

Similar papers

#computer vision Review Sep 2017

Agile Software Development Methods: Review and Analysis

This publication proposes a definition and a classification of agile software development approaches and analyses ten software development methods that can be characterized as being "agile" against the defined criterion.

P. Abrahamsson, O. Salo, Jussi Ronkainen et al. · 727 citations · ⚡54
#computer vision Jun 2008

The impact of agile practices on communication in software development

The study shows that agile practices improve both informal and formal communication, but indicates that, in larger development situations involving multiple external stakeholders, a mismatch of adequate communication mechanisms can sometimes even hinder the communication.

M. Pikkarainen, Jukka Haikara, O. Salo et al. · 401 citations · ⚡48
#machine learning Review Open access Oct 2014

Software development in startup companies: A systematic mapping study

The results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Nicolò Paternoster, Carmine Giardino, M. Unterkalmsteiner et al. · 394 citations · ⚡54

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Oct 29, 2026

Microsoft Research Focus – Edition #97

Host Sowbhagya Suresh shares some of the latest news and cutting-edge research from across Microsoft—from understanding the challenges facing coding agents to uncovering how people use AI at work and improving extended-range weather forecasting. The post Microsoft Research Focus – Edition #97 appeared first on Microsoft Research.

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.