Sep 2026· IEEE transactions on computers· Vol 75, pp. 3055-3069· 0 citations· 43 references
Abstract
Deep Neural Networks (DNNs) have become foundational to intelligent systems, yet deploying them efficiently under strict latency, resource, and privacy constraints remains challenging. While cloud-only inference suffers from transmission latency and privacy risks, and edge-only execution is limited by hardware capacity, cloud–edge collaborative inference offers a practical middle ground by combining the cloud’s compute strength with the edge’s proximity to data sources for low-latency, scalable, and privacy-aware inference. However, realizing this potential requires adaptive DNN partitioning that responds to dynamic workloads and network conditions, as well as fine-grained cross-tier resource orchestration to avoid bottlenecks and ensure system stability. To this end, we propose DDPG-DRPA-driven Deep Neural Network(D3NN), a novel and efficient framework for partitioned DNN deployment across cloud and edge resources. We formulate the pipeline partitioning of DNNs as a Markov Decision Process (MDP). A value function is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm, and a Dynamic Resource Partitioning Agent (DRPA) allocates suitable cloud or edge resources to each DNN layer according to specific task types. As a result, D3NN adapts dynamically to both environmental conditions and task requirements. Under maximum task arrival rate scenarios, our approach reduces inference latency by 13.7% compared to pure cloud-based inference and by 33.5% compared to pure edge-based inference, demonstrating its practical effectiveness in resource-constrained cloud–edge systems.
As deep learning permeates latency-sensitive domains such as autonomous driving and smart surveillance, deploying neural networks (NNs) across the computing continuum (CC), from IoT devices to edge servers and cloud platforms, has become increasingly important. In such heterogeneous IoT-Edge-Cloud environments, distributed inference promises reduced latency, improved privacy, and better resource utilization. Yet, determining how to deploy NNs over heterogeneous IoT-Edge-Cloud nodes remains a difficult and largely manual process. This paper presents a principled and extensible framework for evaluating distributed inference of NNs in heterogeneous CC infrastructures. We introduce a formal model that unifies functional, pipelined, and data-parallel partitioning strategies within a single abstraction over heterogeneous CC topologies, enabling structured cross-strategy comparison. Building on this foundation, we implement a distributed inference orchestrator that supports flexible deployment of partitioned CNNs, and introduce PartiBench, a benchmarking tool that profiles segments and guides their placement. Our evaluation demonstrates how the framework exposes key performance trade-offs, offering actionable insights into latency, memory use, and communication overhead across IoT-Edge-Cloud nodes. These contributions enable empirical, cross-strategy comparison of distributed inference deployments and provide a basis for future automated placement methods in heterogeneous IoT-Edge-Cloud systems.
Nikolaos Papadakis, Alexandros Angourakis, K. Magoutis et al.· International Conference on...· 0 citations
This paper employs five lightweight machine learning models to predict query-specific latency, accuracy, resource usage, and response size for each model-size-quantization-device combination, and design a lightweight heuristic that approximates the BILP solution.
Reza Farahani, Zoha Azimi, Mario Colosi et al.· 0 citations
A scalable edge-to-cloud AI inference pipeline in which inference tasks are dynamically distributed across heterogeneous edge and cloud resources is examined, providing a basis for resilient real-time AI systems while highlighting unresolved challenges involving heterogeneous hardware, dynamic workloads, privacy-utility trade-offs, and cross-layer optimization.
Dr. Khalid Al- Mansour· International Journal of Com...· 0 citations
DABO is proposed, a calibration-aware binary offloading method for collaborative large–small model inference that maintains competitive end-to-end accuracy while processing an average of 83.72% of requests at the edge.
Chen Zhu, Yi-Ming Su, Chenwenjie Mao et al.· IEEE Access· 0 citations
Results show that DEFRAG narrows the SLM-LLM accuracy gap, while reducing cost by up to 98.4% and increasing peak throughput by up to 97.8% over centralized services, demonstrating the potential of DEFRAG for democratized LLM services at the edge.
Jiaxing Li, Hengzhi Wang, Feng Wang et al.· IEEE Transactions on Mobile...· 0 citations
Traditionally, training and hyperparameter optimization of deep neural networks rely heavily on centralized cloud data centers. However, privacy concerns are driving a paradigm shift to move model fine-tuning directly to localized edge environments. Migrating Population-Based Training (PBT) to the edge presents severe challenges: unlike uniform cloud servers, edge networks exhibit extreme hardware heterogeneity. While Low-Rank Adaptation (LoRA) accelerates individual trials on weak devices, permanently maintaining adapters causes structural divergence during PBT weight inheritance. To cope with these edge-specific constraints, we propose Dynamic LoRA-PBT, an asynchronous hardware-software co-design. Systematically, it mitigates extreme evolutionary staleness via a capability-aware scheduler and a late-stage CPU dropout mechanism. Algorithmically, it introduces a Merge-and-Unload strategy, injecting LoRA for early exploration and explicitly merging it into the dense model before mutation. Preliminary evaluations on an 11-node CPU/GPU edge-server cluster demonstrate a 23.4 percent reduction in Time-to-Accuracy (TTA). As an ongoing work, we are currently integrating Transformer architectures and conducting rigorous statistical validations to solidify this framework.
What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduAug 31, 2026
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
MIT News · Artificial Intelligence· news.mit.eduAug 27, 2026
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
