Vertumnus is presented, an adaptive CP serving system designed for heterogeneous and evolving workloads that reduces mean TTFT and improves token-weighted SLO attainment and introduces a global prefix-cache management policy that coordinates cache placement and replication among workers with the same or different CP de...
Jiarui Guo, Rong-Le Wang, Pei-Jun Huang et al.· 0 citations
SCOPE is a training-free sparse attention framework that combines 3D-RoPE-aligned key subspace clustering with online per-head Top-$k$ estimation for efficient video-DiT inference and consistently outperforms existing training-free baselines in both fidelity and latency.
Qi Zhao, Qi-Rui Li, Hanlin Tang et al.· 0 citations
Shiva-DiT, based on Residual-Based Differentiable Top-k Selection, provides four fidelity-latency operating points and reaches a 1.54x wall-clock speedup with competitive fidelity.
C2KV is proposed, a unified framework for non-prefix KV reuse that jointly optimizes KV cache compression and concatenation that significantly reduces KV cache storage and transfer costs.
Chuheng Du, Jun-Yi Chen, Hanlin Tang et al.· Proceedings of the 32nd ACM...· 2 citations
A causal interpretability framework is introduced that reveals that the tokens encoding semantics at the input need not be those that maintain them during generation, but also provides a causal view of internal mechanisms in diffusion transformers.
