The Thinking Reward Model (TRM), a paradigm that explicitly determines what matters for each case before judging how well the candidate performs, is proposed, which formulates case-adaptive rubrics, performs rubric-guided assessment, and produces fine-grained pointwise rewards.
Abstract
Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and candidate outputs directly to scalar rewards, leaving implicit what should be evaluated for each individual case. We introduce Think Before You Score, a paradigm that explicitly determines what matters for each case before judging how well the candidate performs. Following this principle, we propose the Thinking Reward Model (TRM), which formulates case-adaptive rubrics, performs rubric-guided assessment, and produces fine-grained pointwise rewards. We further observe that conventional pairwise preference optimization can induce score polarization, and introduce Pairwise Dual-Group Relative Policy Optimization (PD-GRPO), which leverages pairwise supervision to improve reward discrimination while preserving fine-grained pointwise scoring. Extensive experiments on image generation and editing reward-modeling benchmarks demonstrate that TRM achieves state-of-the-art performance among open-source reward models while remaining highly competitive with proprietary alternatives. Moreover, using TRM as a reward for reinforcement learning consistently improves diverse visual generation models, demonstrating that its fine-grained, case-adaptive rewards translate into effective optimization signals for visual generation.
RewardVerse, a rubric-based video reward framework that introduces a dynamic rubric as an intermediate representation between the evaluation query and the scorer, mitigates scalar drift, and provides a robust and interpretable reward signal for RL in video generation.
Zhen-Chen Tang, Yang Li, Songlin Yang et al.· 1 citation
RubricRM is proposed, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images.
Zi-Jian Kan, Wei Wang, Long Luo et al.· 0 citations
Across six main multimodal reasoning benchmarks and three Qwen3-VL-Thinking model sizes, VIG consistently improves the accuracy--efficiency trade-off, supporting the central claim: efficient multimodal reasoning emerges from raising visual information density.
Wen Luo, Xiao-Han Yi, Xiao-Tao Huang et al.· 0 citations
It is shown that Plackett-Luce (PL) reward models can train robotic policies from VLM-generated rankings as effectively as pairwise Bradley-Terry, $K$-wise Bradley-Terry, and RL-VLM-F baselines and demonstrate that listwise VLM preference supervision is a competitive and flexible approach to reward learning for reinfor...
Srivalli Katkuri, Maxwell Kawada, J. Wachs· 0 citations
A coherent map of the rapidly expanding landscape of visual RL is provided to provide researchers and practitioners with a coherent map of the rapidly expanding landscape of visual RL and to highlight promising directions for future inquiry.
Text-to-image models excel at natural image synthesis but struggle with graphic design, where success depends on satisfying precise constraints on typography, layout, color, and visual communication. While prompt optimization offers an attractive alternative to expensive diffusion model fine-tuning, learning prompts fo...
Adrienne Deganutti, Purvanshi Mehta, Simon Hadfield et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.