Skip to content

Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models

Sep 2026 · arXiv.org · Vol abs/2609.35294 · 0 citations · 47 references
Computer Science

TL;DR

This work proposes a fine-grained alignment framework that couples dense reward signals at the data level with precise credit assignment at the algorithmic level to prevent response-level shared advantages from allowing local hallucinations to compromise all other valid outputs within the same response.

Abstract

Object hallucination remains a major challenge for large vision-language models. While off-policy preference optimization proves to be an effective solution, on-policy reinforcement learning provides a more promising direction as it directly targets a model's current failure modes. However, we find that without fine-grained reward formulation and allocation, on-policy optimization often falls into an easy shortcut: reducing hallucinations merely by saying less---making fewer valid claims. To comprehensively resolve this, we propose a fine-grained alignment framework that couples dense reward signals at the data level with precise credit assignment at the algorithmic level. Specifically, we first construct the Dense Object Presence and Absence (DOPA) dataset to address sparse annotations that prevent valid object claims from being verified and rewarded. DOPA exhaustively annotates the deterministic presence and absence of every concept across an expanded vocabulary, significantly increasing the density of reliable reward signals during on-policy rollouts. Second, we propose Subsentence-level Credit Assignment for on-Policy Optimization (SCAPO) to prevent response-level shared advantages from allowing local hallucinations to compromise all other valid outputs within the same response. By assigning credit to each subsentence independently based on its object claims, SCAPO can precisely reinforce faithful generations and penalize hallucinations. Furthermore, we leverage the resulting faithful image descriptions as auxiliary context to transfer generative gains to discriminative tasks. Experiments demonstrate that our method produces highly informative, faithful descriptions in generative tasks while yielding clear performance gains on discriminative evaluation.

View source

Similar papers

Open access Aug 2026

RoFLIP: Robust and Fine-Grained Alignment for Vision-Language Compositional Reasoning

The Robust and Fine-grained training framework for CLIP-based vision-language models (RoFLIP) is proposed, enhancing both the robustness and granularity of vision-language alignment and underscore RoFLIP’s compositional reasoning and generalization abilities.

Yiwei Sun, Chuan-Bin Liu, Shan-Cheng Fang et al. · 0 citations
Preprint Sep 2026

Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning

The contrastive evidence gap is formalized, the per-token log-likelihood ratio that a model assigns to its own output when conditioned on a question-relevant region versus an irrelevant one, and it is shown that the gap is complementary to the model's own confidence.

Santiram Tiwari, N. Naik, Devbrat Pandey et al. · 0 citations
Preprint Sep 2026

Region-Level Policy Optimization for Fine-grained MLLM Perception

Fine-grained visual perception in MLLMs is commonly improved by raising the resolution, but the added visual tokens inflate vision-encoding and language-model prefilling costs. We show that the two operations underlying fine-grained perception, localizing the region of interest (RoI) and recognizing its content, have d...

Yuheng Shi, Xiao-Huan Pei, Min-Jing Dong et al. · 0 citations
#computer vision Preprint Sep 2026

From Knowing to Abstaining: Bridging the Representation-Action Gap in Vision-Language Models

Rep2Act, a representation-to-action alignment method that translates latent answerability awareness into explicit abstention decisions, is introduced, a representation-to-action alignment method that translates latent answerability awareness into explicit abstention decisions.

Jia-Luo He, Huang-Xun Chen · 0 citations
Preprint Aug 2026

Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding

Clue-OPSD, a clue-privileged on-policy self-distillation framework for long-video understanding that uses clue intervals as privileged supervision without relying on ground-truth answer labels, while requiring no clue annotations or additional modules at inference time is introduced.

Kai-Shen Wang, Dong-Di Zhao, Yijun Liang et al. · 6 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.