Bridging the Spatial Blind Spot in Pathology Foundation Models for Tumor Localization

Abstract

Pathology foundation models (FMs) produce versatile patch-level features but discard inter-patch spatial context, leaving them ill-suited for dense prediction tasks such as tumor localization. Multiple instance learning (MIL) methods offer attention heatmaps as a localization byproduct, yet these maps are optimized for classification and exhibit poor spatial coherence. We present TGSR (Text-Guided Spatial Reasoning), a lightweight cross-modal framework that restores spatial context on pre-extracted features and grounds its predictions in language. TGSR comprises three components: (1) a one-hop spatial aggregation module that reintroduces neighborhood context, (2) a scale-aware router that dynamically balances macro- and micro-level analysis based on local tissue statistics, and (3) an uncertainty-aware two-stage training strategy that mitigates label noise from both ground-truth annotations and MIL-derived pseudo-labels. Classification is performed by cosine similarity against class prototypes derived from LLM-generated morphological descriptions, which makes the semantic target of the model editable at test time. With 527K trainable parameters on 512-dimensional features and no backbone modification, TGSR improves patch-level tumor localization on four datasets spanning three organs, raising Dice by up to 22% over probing baselines under supervised settings and by up to 80% relative over raw MIL attention under weakly supervised settings. Because the semantic target of the model is carried by the text prototypes, a model fitted on lymph node tissue can be applied to colon tissue by replacing the class descriptions, where it exceeds a linear probe trained directly on colon.