AART: image classification with adaptive attention region transformer
TL;DR
The Adaptive Attention Region Transformer (AART) is proposed, which dynamically discriminates between regions based on their saliency, and achieves significant improvements over existing methods without requiring pre-training, validating its effectiveness in adaptive region processing.
Abstract
The Visual Transformer (ViT) has demonstrated powerful capabilities in modeling patch-wise attention for image classification. However, existing approaches typically treat all image regions uniformly, neglecting their inherent differences in importance. To address this limitation, we propose the Adaptive Attention Region Transformer (AART), which dynamically discriminates between regions based on their saliency. Our method begins by identifying key regions through density analysis of feature points, where the centroid of the densest cluster defines attention regions, with remaining areas designated as non-attention regions. We then implement differentiated feature extraction: small convolutional kernels capture fine-grained details from attention regions, while large kernels extract coarse-grained features from non-attention regions. This multi-scale feature extraction strategy enables more efficient representation learning. The resulting features are integrated and processed through Transformer blocks to learn comprehensive self-attentive representations. Extensive evaluations on CIFAR-10 and CIFAR-100 demonstrate that AART achieves significant improvements over existing methods without requiring pre-training, validating its effectiveness in adaptive region processing.