Ronnie Polonsky

Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?

This paper systematically study whether pretraining the Q-function actually helps when fine-tuning on top of a pretrained base policy, and proposes Initialization via Policy Ensemble (IPE), a simple method that trains multiple diverse policies and uses their pooled rollouts to bootstrap the Q-function learning in onlin...