Block-Sparse Attention with Semantic-Geometric Decoupled Routing
Semantic-Geometric Decoupled Routing is proposed, a training-free block routing framework that shifts semantic aggregation to the pre-RoPE space and reconstructs geometric bias with an offline structural prior and relative block distances and yields an explicit closed-form block routing score without token-level search...
Xin-Wei Long, Wei-Gao Sun, Wei-Bo Gao et al.
· 0 citations