LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
Reinforcement learning (RL) has greatly advanced the capabilities of large language models (LLMs), but its memory demands remain a barrier to broader adoption. We introduce LoGRA, an approach to RL post-training that reduces memory by retaining useful learning signals in low-rank gradient sketches. These compact repres...
Shao-Kun Zhang, Yi-Fan Zhang, Jian Hu et al.
· 0 citations