Backbone diversity beats text supervision: a systematic study of frozen multi-foundation model fusion for in-the-wild plant disease recognition
Automated plant disease recognition from field photographs remains challenging: models trained on laboratory datasets collapse on in-the-wild images, and the current state of the art on PlantWild—the largest open in-the-wild benchmark (18,542 images, 89 classes)—relies on text prototypes derived from a language model to reach 76.18% top 1 accuracy. We ask whether text supervision is truly necessary or whether the bottleneck is the diversity of the visual representation. Our central methodological finding is that backbone selection and fusion matter far more than classifier-head engineering: across 118 experiments, a simple fixed-weight linear–prototype combination on top of three complementary frozen backbones yields larger and more reproducible gains than any head-level adaptive routing mechanism we test. Specifically, through a systematic study of six frozen vision backbones (three CLIP, two DINOv3, and one DINOv2), three classifier heads, and four fusion configurations, completed in a single day on one consumer GPU, we establish three findings. (i) A single self-supervised backbone (DINOv2 ViT-L/14) already surpasses text-augmented MVPDR (77.56% vs. 76.18%). (ii) Concatenating three complementary backbones (DINOv2 + DINOv3 + CLIP) reaches 80.23% ± 0.41% (five seeds), exceeding the published MVPDR accuracy by +4.05 points and our own reproduction of MVPDR under an identical evaluation protocol by +7.96 points, without any language supervision. (The difference between the two deltas reflects evaluation–protocol differences—our split, model-selection criterion, and training schedule—rather than any discrepancy in the reported numbers; see Section 4.6.6 for a full reconciliation.) (iii) Every form of learned routing we test—per-class gating, backbone gating, sample-wise gating—is inessential; the gain is entirely attributable to backbone diversity and a simple linear–prototype scoring combination. On the smaller PlantDoc benchmark, the same principle transfers but with substantially higher seed variance: the best configuration reaches 80.09% at a favourable seed but 76.97% ± 1.48% over five seeds—a suggestive rather than robust gain. Beyond the accuracy headline, we provide a pathology-aware per-class analysis showing that DINOv2/v3 dominate on fine-texture lesion classes (rusts, mildews, and leaf spots) whilst CLIP’s narrow advantage concentrates on organ/species-level identification (rice leaf and potato late blight). All primary claims are validated over five seeds, and all code, feature caches, and result files are released for full reproducibility.