Cong-Wei Song

The General Theory of Localization Methods

It is shown that the Transformer -- a cornerstone of modern sequence modeling -- can be constructed using hierarchical local models, revealing the ability of the localization method to unify and generalize state-of-the-art architectures.