Skip to content

Self-Evolving Code-with-Image Reasoning

Aug 2026 · arXiv.org · Vol abs/2608.11292 · 2 citations · 43 references
Computer Science

TL;DR

The Code-with-Image Bench (CwI-Bench), thirty task families induced by hidden visual computations with disjoint learning and evaluation splits, even GPT-5.6-luna stays below 30% with tool-free chain of thought; given a bare interpreter it reaches 43%, and with skills evolved through its own executable reflection, 67%.

Abstract

Multimodal models increasingly reach for tools when solving visual tasks (crop, zoom, rotate, brighten), a paradigm known as thinking-with-images. The central challenge is one of perception: tools mostly serve to expose visual evidence, reasoning over that evidence stays in language, and most targets are ones a human could in principle determine by inspection. Some visual questions, however, are not bottlenecked by perception: recovering their answers requires executing a multi-step visual algorithm over the pixels. On such questions a model often names the correct algorithm at once yet still answers wrong, because language can describe an algorithm without being able to run one. Code-with-Image crosses that line: given nothing but a Python interpreter, the model must implement a genuine visual algorithm in code to solve the task; the program itself becomes the reasoning. The bottleneck then shifts from executing code to deciding which algorithm to implement. So we let the model teach itself: a training-free reflection loop studies its own failed programs, tests repairs against constructive ground truth, and keeps what survives as portable skills. On our Code-with-Image Bench (CwI-Bench), thirty task families induced by hidden visual computations with disjoint learning and evaluation splits, even GPT-5.6-luna stays below 30% with tool-free chain of thought; given a bare interpreter it reaches 43%, and with skills evolved through its own executable reflection, 67%. The open 27B model climbs the same ladder (9% $\rightarrow$ 33% $\rightarrow$ 56%), and the skills are plain text, transferable across scales and families. When code carries the reasoning, debugging code becomes debugging reasoning.

View source

Similar papers

Sep 2026

Reasoning with Image Generation

Across six diverse visual reasoning tasks including multi-view spatial reasoning and collision prediction, ReImaGin consistently outperforms both text-only reasoning and specialist vision-tool baselines, with gains of up to 25\%, demonstrating the advantage of flexible, generative visual reasoning.

Nishad Singhi, Hector P. Rodriguez, Aditya Arora et al. · 0 citations
Preprint Oct 2026

Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning

Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine-grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as...

Ze-Rui Chen, Chang-Rui Chen, Fei Ni et al. · 0 citations
Preprint Oct 2026

DecoyCount: Evaluating Visual-Reasoning Misalignment in Counting

Visual counting benchmarks report whether vision-language models produce the correct number, but not whether errors arise from perception or reasoning. We introduce DecoyCount, a benchmark built from 1,586 real-world images with designed decoys across seven mechanisms, along with paired prompts: a neutral question, a m...

Qiu-Yi Zhang, Zhen Xu, Jing Gu et al. · 0 citations
#computer vision Preprint Nov 2026

See, Think, Use: Diagnosing Perception and Reasoning Failures in VLMs

We present See, Think, Use, a protocol that traces each answer of a vision-language model to perception, reasoning or the use of perception. The diagnosis shows a developer which stage to improve and directs test-time compute to the stage that fails. Benchmarks score the final answer, and an answer that rests on the im...

Jia-Cheng Ding, Yan-Liang Qi, Cong-Yu Guo et al. · 0 citations

MM-OPD: Towards One More Bottleneck Between Perception and Reasoning

MM-OPD is introduced, a multimodal on-policy self-distillation framework for symbolic-to-visual correction that transfers guidance from symbolic-conditioned behavior to the image-conditioned policy through residual token-level targets, steering the model toward correct visual evidence.

Jin-Tao Tong, Yujing Lou, Zhan-Ming Shen et al. · 1 citation
#artificial intelligence Preprint Oct 2026

Where Do Vision Language Models Fail at 3D Reasoning?

Vision-language models can get individual judgments right while struggling to recover and consistently use the exact properties of a 3D object. To characterize this gap, we introduce BedBench, a controlled benchmark of thirteen procedurally generated tasks spanning individual judgments and composite answers. Across twe...

Khai Le-Duc, Nguyen Kim Hai Bui, An Ngo et al. · 1 citation

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.