Diabetic Retinopathy Detection Using Multi-Scale Feature Embedding and Vision Transformer

Abstract

Diabetic Retinopathy (DR) is an important preventable cause of blindness worldwide, thus requiring accurate and early detection methods. Vision Transformers (ViTs) are promising for the task of analyzing retinal images. But the conventional ViT architectures deploy a single classification token ([CLS]) to represent the entire image. This technique may miss the subtle lesion-level attributes that are important for early DR detection. We propose a pure ViT architecture for binary DR classification which is independent of the [CLS] token in this work. Instead of that, it employs mean pooling over all patch embeddings in order to incorporate not only the general retinal context but also detailed local features. We also present a multi-scale patch embedding approach with $8 \times 8$ and $16 \times 16$ patches. This technique has the virtue of capturing small microlesions and larger pathologic structure. An attention-guided token pruning approach is proposed to mitigate the influence of irrelevant background regions. In summary, we have shown that the proposed method is more robust in terms of performance and reliability in binary DR detection providing evidence that this approach may be suitable for clinical retinal screening applications.