Jia-Ni Cao

NPUGen: NPU as KV Generator for Stable Long-Context Mobile LLM Inference

As on-device large language model (LLM) applications like mobile agents become increasingly popular, long-context inference has become a key challenge. The key-value (KV) cache is the foundation of efficient autoregressive decoding, but its memory footprint grows rapidly with context length, often exceeding the memory...