Aug 2026· ACM Computing Surveys· 1 citation· 35 references
TL;DR
A systematic literature review on how RL are adapted and scaled as a fundamental post-training tools and how innovations in the RL pipeline enhance the domain-specific LLMs is conducted.
Abstract
Reinforcement learning (RL), is becoming essential in the post-training of large language models (LLMs), enhancing their capabilities and alignment with human preferences. However, adapting conventional RL to LLMs introduces challenges stemming from their massive parameter size and the vast natural language action space. In this survey, we conduct a systematic literature review on how RL are adapted and scaled as a fundamental post-training tools. First, we provide a taxonomy of challenges faced in each stage of the RL training loop, including action exploration, trajectory collection, reward evaluation, and model update. We then introduce recently developed methods to address these issues, ranging from logical structure navigation, training data curation to reward design and advantage estimation. Afterward, we elaborate on the application of these techniques in diverse domains such as mathematics, coding, medicine, and information retrieval, analyzing how innovations in the RL pipeline enhance the domain-specific LLMs. Finally, we discuss the limitations and side effects of applying RL to LLMs and explore open problems and future directions like the balance between efficiency and effectiveness, and algorithm-system co-design. This survey helps researchers understand recent progress and inspire novel research to address current challenges and realize the full potential of RL for LLMs.
This work deconstructs the RL post-training algorithm, investigating each step to clarify what is actually happening beneath the surface, and uses the entropy of the policy's output distribution as a lens to compare the distributions learned through pretraining, SFT, and RL post-training, revealing how each stage shapes model certainty.
D. Clay, Saket Gollapudi, Sankar V Harilal et al.· 0 citations
This paper highlights the transition from static prediction to sequential decision-making, emphasizing RL’s strengths in long-term reward optimization and interaction modeling, and LLMs’ advantages in semantic understanding and reasoning.
Xi-Qian Lu· Computers and artificial int...· 0 citations
Extensive experiments show that Distilled RL substantially outperforms standard RL and OPD in terms of both pass@1 and pass@k, and can effectively transfer previously unavailable knowledge from a teacher model to a student model.
Chen Wang, Zhaochun Li, Jionghao Bai et al.· 2 citations
This work proposes VEG (verbal ϵ -greedy), a novel framework that leverages external feedback as a dynamic control variable to explicitly balance exploration and exploitation within the semantic space and achieves superior accuracy compared to standard RL baselines.
Yongchang Hao, Jie Hao, Yongsheng Mei et al.· 0 citations
Recent advances at the intersection of reinforcement learning (RL) and Multimodal Foundation Models have enabled agents that not only perceive complex visual scenes but also reason, generate, and act within them. This survey offers a critical and up-to-date synthesis of the field. We first formalize visual RL problems and trace the evolution of policy-optimization strategies from RLHF to verifiable reward paradigms, and from Proximal Policy Optimization to Group Relative Policy Optimization. We then organize more than 200 representative works into four thematic pillars: multi-modal large language models, visual generation, unified model frameworks, and vision-language-action models. For each pillar we examine algorithmic design, reward engineering, benchmark progress, and we distill trends such as curriculum-driven training, preference-aligned diffusion, and unified reward modeling. Finally, we review evaluation protocols spanning policy-level, trajectory-level preference, and training diagnostic stability, and we identify open challenges that include sample efficiency, generalization, and safe deployment. Our goal is to provide researchers and practitioners with a coherent map of the rapidly expanding landscape of visual RL and to highlight promising directions for future inquiry. Resources are available at: https://github.com/weijiawu/Awesome-RL-for-Multimodal-Foundation-Models.
This thesis proposes a unified two-axis framework that organizes SFT and RL methods along a data axis (off-policy to on-policy) and a loss function axis (positive-only to positive-plus-negative to GRPO) and enables controlled ablations of individual components.
G. Kim, Chair Chenyan Xiong, Aditi Raghunathan· 0 citations
