Attention via Black-Box Vector Search
This work unifies prior approaches through the framework of priority sampling, designing algorithms that establish a smooth tradeoff between the number of MIPS indices and number of retrieved keys and designing algorithms that outperform top-$k and sampling approaches used in prior work and yield attention approximatio...