Laurence Aitchison

Le Critique: Privileged Value Functions for LLM Reinforcement Learning

This work proposes two complementary strategies to improve the performance of value function RL: Privileged Value Functions (PVF) which provide an elegant mechanism to inject additional task-relevant token-level signal without biasing the policy objective; and TETHER, a baseline that adaptively interpolates between group-relative and value baselines depending on the value function accuracy.