CALSI: Context-Aware Layer Skipping Inference for On-Device LLM Serving
The large language model (LLM) based on the Transformer architecture and its derived various applications have greatly changed people’s lives. Considering some concerns such as privacy and network conditions, deploying LLM on smart devices has gradually become a research focus. In order to reduce the huge computation a...