CEDAR: Error-Bounded Residual Routing for Efficient Long-Context Attention
Post-hoc sparse attention accelerates long-context prefill by routing each query to a small set of token-level interactions. Hard selection, however, assigns zero probability to every omitted chunk: a routing miss cannot be recovered, and a fixed expansion budget spends the same work on easy and ambiguous queries. We i...
Si-Yu Li, Dong Wang, Jie Zhou et al.
· 0 citations