This work investigates whether curriculum-based prompt selection can improve sample efficiency for Countdown arithmetic reasoning by focusing training on prompts near the model’s current competence boundary, and introduces an adaptive competence-boundary curriculum for RLOO.
This project investigates whether curriculum-based sampling can improve RLOO training by concentrating updates on prompts that are expected to be most informative and suggests a fundamental tradeoff between concentrating updates on potentially informative prompts and maintaining broad coverage of the training distribution.
This work proposes three curriculum variants on top of a RLOO fine-tuning baseline that can concentrate RLOO training on this frontier and improve final performance on the Countdown arithmetic reasoning task.
Darren Chan, Jayna Huang, Sophie Zhang· 0 citations
This work studies two prompt-distribution interventions while keeping the SFT initialization, verifier, and RLOO objective fixed, and compares uniform RLOO, a static curriculum, and an adaptive curriculum.
ADSC is introduced, a curriculum learning layer for REINFORCE Leave-One-Out that uses a signal already computed by RLOO to identify prompts near the student’s current learning frontier and uses a multi-armed bandit router to sample from these difficulty buckets.
Nathaniel Demchak, Pravin Ravishanker, Oscar Li· 0 citations
The novelty is twofold: tying curriculum design to the non-degeneracy of the RLOO advantage, and a zero-overhead adaptive sampler that discovers difficulty from reward rather than from a hand-set proxy.
