Skip to content

MM-OPD: Towards One More Bottleneck Between Perception and Reasoning

Sep 2026 · arXiv.org · Vol abs/2609.32690 · 1 citation · 43 references
Computer Science

TL;DR

MM-OPD is introduced, a multimodal on-policy self-distillation framework for symbolic-to-visual correction that transfers guidance from symbolic-conditioned behavior to the image-conditioned policy through residual token-level targets, steering the model toward correct visual evidence.

Abstract

Recent multimodal large language models (MLLMs) advance visual reasoning by strengthening both perception and reasoning, implicitly assuming a process that transitions seamlessly from perception to reasoning. However, we observe a counterintuitive phenomenon that challenges this assumption: holding the model, question, and decoding fixed, we replace images with their caption or code representations (symbolic views), which seems to be redundant given the clear image structures, but the performance surprisingly improves by 10.2% to 23.6% across model scales and datasets. We term this performance gap as the Symbolic Visual Gap and then take a closer look at it. Through experiments, we find that although the visual evidence can already appear in the reasoning trace for the image-input model, the symbolic-view-input model shows much higher attention to the correct evidence than the image-input model. This suggests that despite good capabilities from current works in perception and reasoning themselves, another bottleneck exists between perception and reasoning in selecting perceived visual information as appropriate evidence for subsequent reasoning. To handle this bottleneck, since the symbolic view steers attention toward correct evidence and is readily obtained at scale, it provides supervision for evidence selection without manually labeled evidence. Building on this, we introduce MM-OPD, a multimodal on-policy self-distillation framework for symbolic-to-visual correction that transfers guidance from symbolic-conditioned behavior to the image-conditioned policy through residual token-level targets, steering the model toward correct visual evidence. Experiments across benchmarks and model scales show that MM-OPD improves a broad range of multimodal abilities, with gains in visual perception, chart and document understanding, mathematical reasoning, and general VQA.

View source

Similar papers

From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models

A small detector is trained to predict the internal development of answer readiness and stop reasoning at that point, connecting the internal development of answer readiness to a practical rule for allocating reasoning computation.

Rong-Yu Xu, Prayag Tiwari, Shao-Lei Zhang · 0 citations
#machine learning Preprint Oct 2026

WOVEN: Weaving Visual World Modeling into Multimodal LLMs

Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect a shared deficit in visual transition reasoning, and test whether this capability can serve as a shared training primitive, one that different models can learn from diff...

Zhe-Yu Fan, Yue Zhang, Ming-Kai Deng et al. · 0 citations
#natural language process... Open access Sep 2026

Func-R1: Incentivizing Mathematical Function Reasoning in Multimodal Large Language Models

Func-R1 is proposed, which synergistically harmonizes precise visual perception and rigorous logical reasoning and delivers the optimal performance among open-source MLLMs, even surpassing GPT-5 with an 8.4% improvement on MathVerse's function-oriented tasks.

Ming Yin, Xiaohai Wang, Dian Li et al. · 1 citation
#artificial intelligence Preprint Sep 2026

UniCAR-RL: Seeing Better before Thinking Deeper in Visual Mathematics

Multimodal Large Language Models (MLLMs) often struggle with complex mathematical visual reasoning primarily due to a lack of fine-grained perception, causing initial visual hallucinations to directly trigger cascading reasoning failures. In traditional end-to-end reinforcement learning (RL), sparse rewards fail to dec...

Yu-Zhe Li, Hao Yan, Hao Wang et al. · 0 citations
#artificial intelligence Preprint Sep 2026

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

It is demonstrated that a pure Large Language Model (LLM), when equipped with specialized modules, can rival state-of-the-art LMMs on complex geometry problems, and achieves performance comparable to Gemini 2.5 Pro while delivering clearer, human-like solutions.

Wei-Chen Dai, R. Cabral, Ziyi Shou et al. · 0 citations
Preprint Oct 2026

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages

Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language reasoning, LR). Reinforcement learning with verifiable rewards typically trains both through a single chain-of-thought with a final-answer rew...

Raja Kumar, Rajat Koner, Ritwick Chaudhry et al. · 1 citation

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.