Ying-Jun Mo

From Policy Evaluation to Off-Policy Control: Provable Emergence of In-Context Q-Learning

Transformers can implement reinforcement-learning algorithms in context, yet how off-policy control emerges within Transformers remains poorly understood. We study Q-learning, a foundational model-free control algorithm that combines off-policy bootstrapping with greedy policy improvement. We prove that a two-layer att...