Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces
The Spectral Pullback Network (SPN) is developed, which learns a low-rank version of the metric from randomized power iteration, and is distill into a $310$K-parameter importance head that predicts token importance directly from the features.
Andrew Bond, Ege Erdem Ozlu, Tuna Çimen et al.
· 0 citations