Large Language Models Develop Belief State Geometry In-Context
Large language models trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly understood, and representation-level evidence that ICL in open-source LLMs approximates optimal Bayesian prediction over a context-inferred generative mod...
Daniel Balcells, Andrew Lee, Chirag Rastogi et al.
· 1 citation
· ⚡1