Skip to content

SAGE: Salient Factor Discovery and Generation with Visual Foundation Representations

Sep 2026 · 0 citations · 43 references
Computer Science

TL;DR

SAGE aims for salient representations that capture target-specific detail in each image, such as the shape, color, and position of the glasses, so that they reveal subtypes without subtype labels and guide the generation of new examples of a discovered subtype, even one with no name or text description.

Abstract

Given a target dataset, such as faces with eyeglasses, and a background dataset, such as faces without, contrastive analysis separates \textit{salient} factors specific to the target from \textit{common} content shared by both. We aim for salient representations that capture target-specific detail in each image, such as the shape, color, and position of the glasses, so that they reveal subtypes without subtype labels and guide the generation of new examples of a discovered subtype, even one with no name or text description. We introduce SAGE, which learns both factors directly in the high-dimensional spatial latent of a frozen representation autoencoder and conditions a diffusion transformer on the learned salient representation of a reference image. On Digits-ImageNet and FFHQ eyeglasses, SAGE combines high-fidelity \textit{reconstruction} (rFID below $2$) with unsupervised \textit{subtype discovery}, recovering the digits better than baselines (probe accuracy $0.950$ vs.\ at most $0.281$) and revealing eyewear types, finer sunglasses styles, and mislabeled images; salient-conditioned \textit{generation} raises Digits-ImageNet subtype accuracy over the unfactorized latent ($90.5\%$ vs.\ $27.7\%$) and diversity on both datasets. On retinal OCT, SAGE's salient space separates three diseases using only normal/disease labels.

View source

Similar papers

Review Open access 2026

A Review of Fine-Grained Visual Categorization with Deep Learning

: Fine-grained visual categorization (FGVC) presents a class of recognition problems in which the discriminative signal is spatially concentrated, visually subtle, and easily destroyed by the preprocessing and augmentation strategies that serve coarse recognition well. Where standard image classification requires a mod...

Richard Adusei, G. Abdul-Salaam · 0 citations
#artificial intelligence Preprint Oct 2026

DisParQ: Self-Supervised Part Concepts for Interpretable Vision Foundation Models

Concept-based vision models represent images through an intermediate layer of human-inspectable concepts, so what a model relies on can be traced to those concepts. However, those models are often limited to fixed categories or depend on language to define their concepts. We introduce DisParQ (Discrete Parts with Quant...

Adam Pardyl, Siddhartha Gairola, Sukrut Rao et al. · 0 citations
Preprint Sep 2026

0.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentation

BRL is proposed, a novel adapter-based PEFT framework that facilitates hierarchical, bidirectional information flow within both token-mixing and channel-mixing layers of frozen foundation models, achieving state-of-the-art performance while requiring less than 0.5% backbone parameter updates.

Xiaoqiang Lu, Li-Cheng Jiao, Ling-Ling Li et al. · 0 citations
#machine learning Preprint Sep 2026

Unifying Distributional Training for One-Step Visual Generation

A unified theoretical framework is introduced that separates distribution modeling from matching discrepancy and connects global objectives to pointwise feature updates through Wasserstein gradient flow and motivates MGFlow, which models feature distributions with Gaussian mixtures at an adjustable granularity between...

Chi Zhang, Hao-Yan Shi, Yue-Yi Liu et al. · 2 citations
Preprint Sep 2026

MINER: Multi-crop INference-time Enhancement for Rare-Object Retrieval with Frozen Dual Encoders

Text-to-image retrieval with frozen dual encoders degrades when the query names a small, visually subordinate object in a cluttered scene: a single global image embedding underrepresents the localized visual evidence. We present MINER, a training-free inference framework that augments a frozen dual encoder's global ima...

Abdulmalik Alquwayfili, Faisal Almeshal, Jumanah Almajnouni et al. · 0 citations
Sep 2026

UnfoldCRF: Structured Mask Refinement with Image-Conditioned Latent Regions

UnfoldCRF treats refinement as inference in a conditional random field over pixel labels and latent region variables, and zeroing individual messages shows where the corrections come from: the pairwise messages mostly fix boundaries, the region messages mostly fix non-boundary errors.

Chun-Ming He, Rihan Zhang, Lei Xu et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

Microsoft Research Blog Aug 11, 2026

Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement

Radiology AI is evolving beyond report generation. CARE-X explores a unified approach that combines flexible reasoning, calibrated predictions, and measurement-based tools for chest X-ray interpretation. The post Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.