Metric Knowledge Distillation for Zero-Shot Cross-Domain Fine-Grained Pill Identification

Abstract

Pill recognition is hard across regions: training and deployment images differ across regulatory standards (e.g., U.S. FDA vs. Hungary’s OGYEIv2), and smartphone-budget compact models often lose accuracy on out-of-region pills. We benchmark 31 deep architectures on a U.S. source dataset (ePillID) and evaluate zero-shot transfer to a Hungarian target (OGYEIv2), observing a capacity–robustness trade-off: heavyweight CNNs lead on the source split but lean on texture cues that do not transfer, so their cross-region accuracy can fall below a distilled compact student. We propose feature-based knowledge distillation with an angular metric-learning head: a Cosine Embedding loss aligns student and teacher L2-normalized embeddings, while an ArcFace margin keeps fine-grained classes separable. The strongest pair (ConvNeXtV2-Base into ShuffleNetV2) gains +0.1433 mAP on ePillID; under zero-shot transfer to OGYEIv2, the distilled ResNet18 reaches 0.9018 Rank-1, +0.0328 above its ResNeSt101e teacher. Two negative-transfer cases and ViT non-convergence under the shared recipe are analyzed, showing distilled students can be lighter and more transferable than their teachers.