Selective Confidence-Guided Projection-Based Encoding for Medical Image Classification

Abstract

Deep neural networks have achieved strong performance in medical image classification, but their deployment may be constrained by the computational cost of high-capacity models. Knowledge distillation (KD) addresses this problem by transferring knowledge from a teacher to a lightweight student. However, the reliability of teacher supervision may vary across samples, potentially introducing noisy guidance and local conflicts with ground-truth supervision. We propose Selective Confidence-guided Projection-based Encoding (SCOPE), a conflict-aware KD framework comprising Selective Relation Alignment (SRA) and Gradient Conflict Resolution (GCR). SRA constructs reliability-aware relational supervision by combining teacher-derived relations with dataset-specific auxiliary priors, whereas GCR removes distillation-gradient components that conflict with the classification objective. Experiments on nine medical image datasets and multiple teacher–student architectures demonstrate competitive predictive performance, improved training stability, and low computational overhead.