LLM Deployment Strategies on Mobile Edge Servers for Dynamic Uncertain User Requests

Abstract

Leveraging on the task planning and solving capability of pretrained Large Language Models (LLMs), deploying LLM agents on Mobile Edge Computing (MEC) edge servers brings significant benefits for an Internet of Things (IoT) network for providing enhanced AI intelligence with acceptable delay. In this work, we consider the edge LLMs deployment strategy in an end-edge-cloud LLM agents system for the IoT services, which jointly determines the locations and number of LLM initializations and user requests offloading strategy in a dynamic network environment with stochastic user requests. We formulate this joint LLM Deployment and inference Tasks Offloading (LLMDTO) problem. Typically, we design an LLM service performance evaluation mechanism by measuring its processing delay with stochastic user requests arrivals by Stochastic Network Calculus (SNC). Due to the complexity of the LLMDTO problem, we decompose this joint optimization problem into two subproblems and propose an algorithm based on Multi Agent Deep Reinforcement Learning (MADRL) scheme. To accelerate the training process of the DRL, a reward model is designed by applying the Kolmogorov Arnold Networks (KAN) to return a fast reward estimation. Finally, we validate the proposed algorithm through extensive simulations and results show the effectiveness of the proposition on lower deployment cost and delay in a dynamic network environment.