Improving Parameter-Efficient Medical Image Classification with Lesion-Aware Hierarchical Knowledge Distillation

Abstract

Compact deep models are attractive for medical image classification, but conventional knowledge distillation mainly transfers class-level predictions and may not adequately preserve lesion-relevant spatial cues. To address this limitation, we propose LaHKD, a lesion-aware hierarchical knowledge distillation framework for parameter-efficient medical image classification. LaHKD enables a lightweight student to learn hierarchical semantic representations together with lesion-focused guidance from a stronger teacher. We evaluate LaHKD on HAM10000 dermoscopic lesion classification and a brain tumor MRI classification benchmark. Across both datasets, LaHKD improves compact-student classification performance, with the clearest lesion-focused spatial benefits observed on HAM10000, where lesion morphology is central to diagnosis and direct lesion supervision is available. On the magnetic resonance imaging (MRI) benchmark, localization analysis is limited to an auxiliary recovered-mask subset and is therefore interpreted as exploratory; under this setting, consistent localization gains are not observed. Overall, LaHKD provides an effective framework for compact medical image classification, with spatial benefits most clearly supported in tasks with reliable lesion supervision.