Yeohn-Shik Choee

In-Browser Split Inference for Autoregressive Large Language Models With Activation Quantization

splitLLM, a novel zero-installation system that partitions autoregressive LLM inference between a WebGPU browser client and remote server, maintaining a key–value cache on both sides of the partition, offers a promising platform for enhancing the accessibility of LLMs, especially for resource-constrained client devices...