WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing
The introduction of WhiteMatter, which allows every layer to draw on past-token representations from any depth, and cyclic iteration, which updates interleaved groups of tokens in turn while processing the tokens within each group in parallel.