Jul 2026
Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection
This work shows that classical data compression provides an effective masking signal with no additional parameters, and identifies non-redundant content blocks and route long-range attention selectively through them by computing per-block gzip compression ratios.
Debarshi Kundu, Swaroop Ghosh, Vasant Honavar
· arXiv.org · 0 citations