Parabolic Positional Encoding: A Vision-Centric, Principled, Extrapolatable, and General Approach
Parabolic Positional Encoding is proposed, a parabola-based positional encoding for vision modalities in attention-based architectures designed from principles distilled from prior work: translation invariance, rotation invariance, distance decay, directionality, and context awareness.
Christoffer Koo Øhrstrøm, Rafael I. Cabral Muchacho, Yifei Dong et al.
· 0 citations