Interpretable Brain-computer Interface for Emotion-aware Thought-to-speech using Eeg Signal Decoding

Abstract

Restoring effective communication for people with severe speech impairment remains a most important concern in assistive healthcare. Electroencephalography (EEG)-Based Brain–Computer Interfaces (BCI) can decode imagined speech from neural activity; however, existing techniques have been limited by a lack of interpretability, reduced robustness across users, and inadequate integration of emotional context. In this study, we present an Interpretable Brain–Computer Interface for Emotion-Aware Thought-to-Speech that incorporates affective state recognition, EEG signal decoding, and explainable deep learning within a unified framework. The presented framework combines advanced EEG preprocessing, spatio-temporal feature representation, attention-guided neural learning, and an explainability module while identifying the underlying neural features and cortical regions influencing each prediction. Emotional state estimation is combined with speech intention decoding to produce speech that reflects contextual expressiveness, enhancing the naturalness and effectiveness of human–computer interaction. The framework is tested on publicly available EEG datasets of imagined speech, using subject-independent setups, and evaluated based on classification accuracy, precision, recall, F1-score, inference speed, and interpretability. Results show that the proposed approach consistently enhances decoding reliability and offers clear, clinically meaningful insights compared to traditional opaque models. By simultaneously addressing speech intent, emotional nuance, and model transparency, this study contributes to the advancement of trustworthy, next-generation brain–computer interfaces and provides a practical basis for assistive communication tools serving individuals with paralysis, amyotrophic lateral sclerosis, locked-in syndrome, and other conditions that disrupt natural speech.