RouteRelay: Event-Triggered Cross-Layer Route Reuse for Efficient Dynamic Sparse Attention
This work introduces RouteRelay, a router-agnostic method that reuses only route metadata across depth while continuing to compute attention with the current layer's queries, keys, and values and gives a top-$k$ stability condition, a probabilistic bound on missed challengers, and a row-selective GPU execution design.
Bin Li, Si Liu, Chen-Yang Hu et al.
· 0 citations