Skip to content
Preprint

Multi-scale Fusion of Fast Vision Mamba and Foundation Model Features for Patch-Level Gleason Grading in Computational Pathology

Oct 2026 · 0 citations · 22 references
Computer Science

TL;DR

This work proposes a multi-scale classifier built on state-space models that combines two complementary feature sources: a frozen foundation model, and a lightweight FastVim, pre-trained on ImageNet and distilled from the foundation model, that reads the raw image crops.

Abstract

Automated Gleason grading of prostate whole slide images (WSIs) enables faster and more consistent cancer risk stratification. However, it remains a challenging task due to the subtle morphological differences between grades, the severe class imbalance arising from rare patterns, and the diagnostic features that span multiple spatial scales. Pathology foundation models (FMs) extract strong patch representations but often operate at a single-scale, which limits spatial context and disproportionately affects classes that depend on broader context. The challenge of fusing multiple scales and complementary models, with minimal data and computational resources, remains unresolved. We propose a multi-scale classifier built on state-space models. Our method combines two complementary feature sources: a frozen foundation model, and a lightweight FastVim, pre-trained on ImageNet and distilled from the foundation model, that reads the raw image crops. At each scale, a Mamba model encodes the features from both sources with linear-time spatial context, and a jointly trained head fuses the encoded representations into one prediction. On the PANDA dataset, our method reaches macro F1 of 0.894 and Gleason 5 F1 of 0.802 under 5-fold cross-validation, a 12% and 21% relative improvement over the best single-scale model. A controlled study over 5 foundation models, 5 base classifiers and 3 scales confirms that the gain holds across encoders, with larger improvements observed at smaller scales.

View source

Similar papers

Open access Oct 2026

Multi-Stage Feature Fusion for Breast Cancer Histopathology using EfficientNetV2 and Vision Transformers

Early detection of breast cancer is essential for improving patient survival and reducing mortality. Although histopathology-based diagnosis provides definitive clinical evidence, the complex and heterogeneous nature of breast cancer histopathology (BCH) images poses significant challenges for accurate classification....

Errissya Rasywir, Yovi Pratama, Yessi Hartiwi et al. · 0 citations
Preprint Oct 2026

LEAF-Mamba: Feature Adaptation and Fusion for Whole Slide Image Analysis

Whole slide image (WSI) analysis enables computational diagnosis and prognosis, but gigapixel images and limited annotations make slide-level learning challenging. Multiple instance learning addresses this task by aggregating patch representations from frozen pretrained encoders. Recently, Mamba has enabled efficient m...

Lu-Bin Gan, Jing Zhang, Ming-Jun Ma et al. · 0 citations
Sep 2026

Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathology

CRAFT (Coarse-to-fine Region-Adaptive Feature Tokenization), a DINO-based framework that learns image-dependent mixed-scale representations by using self-supervised attention to selectively refine informative regions while preserving coarse context, together with a symmetric cross-scale regularization objective that en...

Anabel Stammer, Valay Bundele, Mehran Hosseinzadeh et al. · 0 citations
Open access Sep 2026

MedFuse: dual-stream fusion of convolutional and vision transformer-based features for enhanced medical image classification

Introduction Medical image classification is fundamental to computer-aided diagnosis. Limited labeled samples, subtle inter-class differences, and heterogeneous lesion morphology make it difficult for a single representation to capture all relevant cues. This study investigates MedFuse, a simple dual-stream framework t...

Ya-Jing Ren, Hai Ling, Zheng Gu et al. · 0 citations
Open access Oct 2026

SkinFusionNet: Multi-backbone spatial fusion for robust and explainable melanoma classification.

Automated melanoma detection from dermoscopic images remains a critical yet unresolved challenge due to substantial dataset heterogeneity, artifact contamination, and limited model generalization across clinical sources. Existing deep learning methods commonly rely on single-dataset training, single-backbone architectu...

Faysal Ahmmed, Resadus Salehin Rafsan, Muhtadi Mansib et al. · 0 citations
Sep 2026

Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images

SlideTIM is proposed, an adaptation to WSIs of the recent transductive approach LC-TIM, which introduces a combined spatial--latent regularizer together with a prior on the patch class distribution that addresses the complex spatial organization and the strong class imbalance of WSIs.

T. Godelaine, M. Dausort, Karim El Khoury et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.