Preprint
Jul 2026
COBS: Cumulant Order Block Sparse Attention
COBS (Cumulant Order Block Sparse Attention), an attention method that builds on NSA, incorporating a novel selector that stores a compressed second-order statistic per block, which achieves lower position-wise negative log-likelihood (NLL) than dense attention in the authors' comparison.
Alexander Tian, Aditya Ghai, S. Neelam et al.
· 1 citation
· ⚡1