Vision Transformer–Driven Feature Extraction for Explainable Pneumonia and COVID-19 Detection from Chest X-Ray Images

Abstract

In order to diagnose respiratory disorders such as pneumonia and COVID-19, X-ray imaging of the chest is essential. On the other hand, radiologists could differ in their approaches and the amount of time it takes to manually analyze radiographic pictures. Recent innovations in deep learning have greatly enhanced automated examination of medical pictures, although conventional convolutional neural networks often miss long-range spatial interactions in complicated pulmonary patterns. To overcome this drawback, this work suggests a Vision Transformer (ViT)-based feature extraction system embedded in the GenMAT-Net framework to classify X-ray images of the chest automatically. The proposed method initially achieves lung region segmentation to isolate clinically significant regions of the image. After that, the pictures are divided they are converted into patch embeddings after being converted into fixed-size patches. To keep the spatial relationships, they are augmented using positional encoding. To acquire global contextual information and multifarious feature linkages throughout the lung regions, these embeddings are trained using numerous layers of transformer encoders that consist of multi-head self-attention and feed forward networks. The semantic features representations at high-level are then optimized and used to classify the disease into three categories, namely, normal, pneumonia, and COVID-19. As shown by experimental assessment, feature extraction based on transformers offers better contextual representation of the lung abnormalities, which allows the effective identification of the pathological patterns, i.e., ground-glass opacities, consolidations, and diffuse infiltrates. The proposed framework also embraces interpretability based on attention mechanisms, where visualization of regions leading to the diagnostic decision can be done. In general, the technique for extracting features based on the application of the Vision Transformer can increase diagnostic reliability and provide a promising solution to intelligent computer-aided diagnostic systems in the medical imaging field.