This work introduces ThunderSyncRL, which starts gradient computation as soon as all required inputs are fixed, without policy staleness, and proves that gradient streaming produces the same GRPO and OPD updates as batch-synchronous training, without changing either objective.
Seil Kang, Hangoo Kang, Tarun Suresh et al.· 0 citations
An emerging hypothesis about reinforcement learning (RL) post-training of large language models (LLMs) is that it merely sharpens existing behaviors of a base model, improving single-shot accuracy at the cost of solution coverage. Although this trade-off has been observed in math and coding tasks, it need not extend to...
To scale BP to long contexts, this work introduces context-sharded block parallelism (CSBP), which keeps corrupted K/V and gradients local, avoids replicated clean prefixes, and preserves BDLM training semantics.
Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang et al.· 0 citations
A notable byproduct of LLM alignment training is mode collapse: the progressive loss of output diversity that narrows a model's expressivity at inference time. This degradation is especially limiting for applications requiring open-ended exploration and pluralistic perspectives, such as scientific ideation and creative...
Jia-Yi Yuan, Hangoo Kang, J. Liu et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.