Aug 2026· Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing· Vol abs/2608.25580· 1 citation· 73 references
Computer Science
TL;DR
This work introduces Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness, Reasoning Consistency, and Instruction Following, and shows rubrics as a useful reward abstraction for visual post-training.
Abstract
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
Savor is introduced, a training framework that augments the output schema with token and answer confidence, optimises the policy with a Group Relative Policy Optimisation objective that penalises calibration error and poor abstention decisions, and uses the learned confidence at inference time to revisit visual evidenc...
Zian Ding, Zi-Lin Zhao, Ying-Jie He et al.· arXiv.org· 0 citations
Persistence-Aware Credit Gating (PACG), which attenuates positive credit for unusually persistent visual claims and leaves all other credit unchanged, which extends to a larger model, a newer backbone, and the accuracy of HallusionBench also improves consistently.
GRIT, a training framework that incorporates visual grounding into RLVR to mitigate the impact of Correct-but-Visually-Misgrounded Reasoning (CVMR), is proposed and introduced, and experiments demonstrate that GRIT significantly improves both reasoning accuracy and grounding fidelity over existing RLVR methods.
Chanseul Cho, No-Chan Bae, Dongheon Lee et al.· 0 citations
RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Rel...
Hang Ran, Yu-Hao Zheng, Ying-Ying Zhang et al.· Conference on Empirical Meth...· 1 citation
Long chain-of-thought reasoning improves vision-language models (VLMs), but it also exposes a temporal grounding failure: as generation unfolds, models may drift from image evidence while reinforcement learning still provides only a final-answer reward. We study this gap through visual revisit, a temporally localized r...
Zhaolu Kang, Tai-Long Luo, Chen-Xin Li et al.· 0 citations
Rubric-based reward models (RMs) evaluate responses against explicit criteria. However, existing methods often derive criteria from observed response candidates, introducing response-induced bias that may limit rubric generalizability. Their judges also struggle with mechanical constraints when relying on reading alone...
Tianci Liu, Hao-Yu Wang· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.