Hong-Yuan Gu

How Many Thoughts Can a Vector Hold? The Capacity of Reasoning by Superposition

Large language models solve hard problems through intermediate computations across multi-step reasoning. Traditional chain-of-thought encodes these computations as tokens. Recent continuous and recurrent methods instead move partial computations into fixed-dimensional latent states, where a single thought can superpose...