XTrust Vision An Explanation-Guided Reliability Assessment Framework for Trustworthy Vision Transformers

Abstract

ViTs (Vision Transformers) have been developed as many of the most popular architectures in contemporary computer vision. They harness self-attention strategies to capture long-range dependencies of images. This allows them to achieve learning of rich global contextual information that is very challenging to model with classical convolutional networks. As a consequence, ViTs have performed so well for a range of image recognition tasks. However, despite their success, there is still worry about the reliability and trustworthiness of their predictions. It’s these concerns that become crucial in applications that depend on open and clear accountability in the decision-making aspects. Current Explainable Artificial Intelligence (XAI) methods mainly concentrate on visual explanations of model behavior. Nevertheless, the reliability evaluation often depends on prediction confidence scores. The confidence values may not reflect the best description of the performance or reliability of the explanations made. As a result, there should be extensive trust evaluation approaches. In this study, we introduce XTrust-Vision for explaining reliability in Vision Transformers to tackle this problem. The framework combines Prediction Confidence (PC), Explanation Consistency (EC), and Attention Alignment (AA). Combining together all three of these components gives a unified Trust Reliability Index (TRI). The TRI offers a more holistic picture of prediction trustworthiness. The experiments done on a 5,000-image set of the CIFAR-100 dataset with a ViT-Tiny model brought about 70.00% accuracy, 73.38% precision, 70.00% recall, and a 70.25% F1-score. The results indicate that by bridging explainability and reliability, tests can inform the building of computer vision technologies that are increasingly transparent, interpretable, and trustworthy.