Skip to content
Log in
Review

Let Me Explain!

Hadi Mohammadi
· 0 citations

TL;DR

Text content is the dominant factor in annotation decisions, far outweighing annotator demographics, and that content-focused SHAP explanations are more effective than demographic persona prompting for guiding LLM annotations, showing that explainability methods can improve both the reliability and the transparency of NLP systems.

Abstract

Large Language Models (LLMs) have transformed natural language processing (NLP), but their billions of parameters make them opaque. This lack of transparency is especially problematic in high-risk areas such as healthcare, finance, and content moderation, where understanding model decisions is essential for responsible use. This dissertation focuses on Explainable NLP (XNLP), making NLP models transparent, as a fundamental requirement for trustworthy AI systems. The dissertation addresses four research questions. RQ1 asks how XNLP methods can be designed and applied to meet the unique demands of high-risk domains such as healthcare, finance, and social media moderation. RQ2 investigates how token-level explanation methods can provide transparency in text classification systems and reveal vulnerabilities to adversarial manipulation. RQ3 examines the extent to which annotator demographics influence labeling decisions and how content-driven XAI techniques compare to demographic persona prompting for LLM-based annotation. RQ4 explores how moral alignment in LLMs can be evaluated across cultures in a systematic and transparent manner. To address these questions, the dissertation employs token-level explanation methods such as SHAP and LIME across multiple tasks. It surveys XNLP applications across domains, identifying gaps between methodological research and practical deployment; develops a transparent sexism-detection pipeline that combines BERT-based classification with SHAP explanations so that content moderators can verify decisions at the word level; and applies explainability methods to AI-generated text detection, revealing that detectors often rely on superficial features and showing that token-replacement experiments can expose these vulnerabilities and improve robustness. The dissertation then shifts to human-centered evaluation and moral alignment. It finds that text content is the dominant factor in annotation decisions, far outweighing annotator demographics, and that content-focused SHAP explanations are more effective than demographic persona prompting for guiding LLM annotations. It evaluates how well LLMs capture moral attitudes across cultures, finding that instruction-tuned models achieve moderate alignment with human survey data but show a persistent Western-centric bias. Finally, it introduces the EvalMORAAL framework, which combines Chain-of-Thought (CoT) reasoning with LLM-as-judge peer review; explicit reasoning consistently improved alignment compared with implicit scoring, though a significant gap between Western and non-Western regions remains. Together, these contributions show that explainability methods can improve both the reliability and the transparency of NLP systems. The dissertation also acknowledges limitations, including a focus on classification tasks and mainly English text, and limited human evaluation. It outlines a vision where explanations are actively integrated into model training, creating a feedback loop between human evaluation and model improvement, and the persistent regional difference in moral alignment underscores that making AI systems transparent and fair is an ongoing effort that requires continued attention to cultural diversity.

DOI Semantic Scholar
View source

Similar papers

Preprint Jul 2026

Evaluating Large Language Models on Misconceptions in Multi-Turn Medical Conversations

Patients seeking medical information often ask questions that embed incorrect assumptions or misconceptions. In such cases, safe medical communication requires not only answering the question, but identifying and correcting the underlying false belief. These interactions naturally unfold over multiple turns, a pattern now mirrored in interactions with LLMs. Yet current evaluation frameworks do not capture model behavior in these settings, where misconceptions can emerge, persist, or evolve over the course of a conversation. Whether LLMs can reliably correct such misconceptions over time remains largely unexamined. To study this, we introduce ThReadMed-QA, a multi-turn medical dialogue dataset of 2,437 patient-physician conversation threads comprising 8,204 question-answer pairs, derived from real patient interactions on AskDocs. This dataset enables systematic evaluation of whether models can detect and correct misconceptions under a multi-turn context. We evaluate five LLMs using a rubric-based LLM-as-a-Judge framework that scores responses based on their ability to identify and correct misconceptions. Our experiments reveal a consistent pattern: even frontier models that can address misconceptions in a single interaction degrade substantially over subsequent turns. GPT-5 and Claude-Haiku correct these false presuppositions around 85% on initial questions but drop to roughly 50% within two follow-ups. An oracle analysis replacing prior model outputs with physician responses shows that much of the degradation is driven by error propagation, while performance remains imperfect even under correct context. Even when models tend to correct misconceptions initially, their performance degrades substantially over later turns, leading to inconsistent and potentially unsafe guidance in patient-facing settings and highlighting the need for evaluation frameworks that capture multi-turn behavior.

Monica Munnangi, Saiph Savage · 0 citations
Review Open access Aug 2026

Explainability of decoder-only clinical large language models: A scoping review.

Findings show that clinical LLM explainability has shifted toward fluent generative rationales, but evidence that such explanations reflect model reasoning remains limited, and three regulatory priorities are highlighted: prioritizing explanations that enable independent verification or logic auditing over plausibility-only rationales; preferring inspectable models where regulatory documentation is required; and prospectively validating explanations in clinical workflows before scaling.

Nishant Mishra, Ameen Abu-Hanna, Iacer Calixto · 0 citations
Preprint Jul 2026

Inference-Time Mitigation of Adversarial Political Bias in Large Language Models

The proposed Recursive Self-Correction approach raises model performance from a Political Neutrality Likert scale baseline of 2.14 to 4.56, averaged across all models, demonstrating effective inference-time mitigation of political bias in LLM-generated summaries.

Tejaswi V. Panchagnula, Bruce Coburn, Bryce J. Dietrich et al. · 0 citations
Review

A Survey on Actionable Interpretability in Large Language Models

This survey reviews LLM interpretability through the lens of actionability, presenting a taxonomy of attributional and mechanistic approaches, along with emerging methods tailored to vision–language models (VLMs), and examining how actionable interpretability supports downstream objectives.

Jie Cai, Mafizur Rahman, James Enouen et al. · 0 citations