Grounding-Aware Self-Distillation for Visual Reasoning

Abstract

Self-distillation reinforcement learning usually selects teacher traces by answer correctness. In multimodal tasks, that criterion is incomplete: a model can answer correctly from language priors without using the image, and distilling such a trace reinforces ungrounded reasoning. We introduce visual counterfactual sensitivity (VCS), the length-normalized log-likelihood drop of an existing trace when its image is corrupted. VCS requires one additional teacher-forced forward pass per correct candidate trace, no additional generation, and no external critic. GRACE uses VCS to select among already-correct traces rather than adding it to the reward, avoiding an incentive to inflate the score through verbose image descriptions. With Qwen3.5-9B, GRACE achieves 66.8% average accuracy across the four Bongard-HOI splits, compared with 61.4% for GRPO and 63.1% for a DRIFT reproduction. The gain is largest on the unseen-action and unseen-object split, where language priors are least useful. HallusionBench improves from the published 69.3% to 72.6% and POPE from 88.9% to 92.4%. Treating VCS as a reward produces a 0.41 correlation with trace length, versus 0.09 under gating. These results support target selection as a distinct control point for multimodal self-distillation.