Collaborative Large Model Caching and Inference Offloading With Parameter Sharing in MEC

Abstract

Pretrained Foundation Models (PFMs) enable highaccuracy inference services but are typically deployed in remote datacenters, resulting in prohibitively high inference delay. Mobile Edge Computing (MEC) can mitigate such high delays by caching PFMs or their fine-tuned variants on cloudlets located close to end users. However, the substantial resource requirements of PFMs and the significant parameter redundancy among their fine-tuned variants make it impractical and uneconomical for resource-constrained cloudlets to cache every large model independently. Motivated by this critical issue, we investigate the collaborative large model caching and inference offloading problem in an MEC network driven by a service market consisting of multiple selfish service providers. The novelty of our study lies in exploring parameter sharing among large models to jointly optimize inference accuracy, delay, and cost while maximizing the total payoff of the service market. To this end, we propose a coalition formation mechanism that enables economic cooperation and fair payoff allocation among different cloudlets. Specifically, we first formulate the optimization problem as a Non-Linear Integer Program (NLIP), and then reformulates it into an equivalent and computationally tractable Integer Linear Program (ILP). We then develop a resource-aware randomized algorithm with a provable approximation ratio. We further incorporate an ε-perturbed best-response process to ensure that the proposed mechanism eventually converges to a stable coalition structure. Experimental results on a real-world dataset demonstrate that the proposed algorithm reduces the average delay and cost by at least 19% and 9%, respectively, while achieving a 10% increase in the total payoff.