Skip to content
Book Open access

Evolving, Not Training: Training-Free Reasoning Segmentation via Evolutionary Prompting

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 34 references

Abstract

Reasoning Segmentation requires models to interpret complex, context-dependent linguistic queries to achieve pixel-level localization. Current dominant approaches rely heavily on Supervised Fine-Tuning (SFT) or Reinforcement Learning (RL). However, SFT suffers from catastrophic forgetting and domain dependency, while RL is often hindered by training instability and rigid reliance on predefined reward functions. To break the bottleneck of static inference paradigms, we propose EVOL-SAM3, a zero-shot framework that reformulates reasoning segmentation as a dynamic inference-time optimization problem. A fundamental challenge in this formulation is the non-differentiable modal gap between discrete semantic reasoning and continuous pixel-level execution. EVOL-SAM3 iteratively refines prompt populations via a Generate-Evaluate-Evolve loop, employing a Visual Arena for tournament-based fitness assessment and Semantic Mutation for error correction. A Heterogeneous Arena integrates geometric and semantic priors to ensure robust selection. Extensive experiments demonstrate that EVOL-SAM3 not only outperforms baselines in terms of primary metrics and token consumption, but also surpasses fully supervised state-of-the-art methods on the ReasonSeg benchmark in a zero-shot setting.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.