Aligning Training and Rollout Policies for End-to-End FP4 RL on Mixture-of-Experts LLMs
In this work, we study end-to-end FP4 reinforcement learning (RL) post-training for Mixture-of-Experts (MoE) large language models. Recent work on FP4 RL has introduced effective recovery recipes that restore accuracy under aggressive low-precision quantization for dense models, but we find that the same approach yield...
Hei-Yi Mak, H. Le, Mehran Taghian Jazi et al.
· 0 citations