Yu-Chien Liao

Inference-Time Compute Allocation in LLMs: Dynamic Depth and Token Selection for Efficient Problem Solving

Large language models (LLMs) deliver impressive performance across diverse natural language processing tasks, but their intensive computation and memory requirements severely restrict real-world deployment, especially on edge devices and in latency-sensitive cloud services. Static inference pipelines execute full layer...