Xiao-Yu Li

One Spectrum, Two Resources: Data-Memory Scaling in Autoregressive Prediction

How much learned memory is needed to benefit from more data? We show that the two resources are governed by one predictive-energy spectrum in a positive-entropy autoregressive retrieval source. Each coordinate contributes its query probability times the squared radius of its unknown logit. Writing $\mu$ for the resulti...