Kalman Delta Networks: Uncertainty-aware Associative Memory
This work introduces Kalman Delta Networks (KDNs), a family of linear-attention models that explicitly track memory uncertainty to guide each update, and derives Diagonal KDN and Isotropic KDN, which retain one uncertainty value per key channel and per head, respectively.
Ngoc Bui, Ting-Lin Huang, Rex Ying
· 1 citation