Explainability and Reliability in Large Language Model Systems: A Survey of SHAP, LIME and Attention Frameworks

Abstract

Increasing adoption of language models in sensitive sectors such as healthcare, finance, cybersecurity, education, etc., highlights the significance of developing transparent decision support systems. This paper reviews the use of explanation techniques for language and machine learning models, including SHAP, LIME, attention-based explanations and frameworks that transform outputs from models into natural language explanations. It turns out that the existing literature demonstrates that although classical language models demonstrate good predictive and semantic performance, they lack transparency. In this regard, SHAP and LIME help in the detection of feature importance and combination of frameworks lead to explanations being more understandable for users. The literature also outlines some limitations of the existing techniques such as explanation faithfulness, hallucinations, biases, privacy issues, computational complexity and lack of benchmarks for evaluation. The use of explainability in healthcare, finance, cybersecurity, industrial systems and IoT is discussed based on the evidence found in the literature.