Deep Learning Approach for Facial Expression Identification Using Hybrid CNN-LSTM Architecture with Attention Mechanism
Abstract
-Facial emotion recognition (FER) has emerged as a critical area of research in computer vision and affective computing, with applications spanning healthcare, human-computer interaction, security, and education. This paper presents a hybrid deep learning framework combining Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks for accurate and robust facial emotion recognition. The proposed system utilizes MobileNetV2 for efficient spatial feature extraction and a 128-unit LSTM layer with temporal attention for modeling sequential dependencies in facial expressions. A curated dataset of 7,500 facial images across seven emotion classes (Angry, Disgust, Fear, Happy, Neutral, Sad, Surprise) was employed, with 5,500 images for training and 2,000 for validation. The system was implemented using Python with TensorFlow 2.x and Keras API, deployed on an NVIDIA Tesla T4 GPU. Experimental results demonstrate superior performance, achieving an overall accuracy of 97.8%, macro-average precision of 97.6%, recall of 97.4%, and F1-score of 97.5%, with an AUC-ROC of 0.982. Class-wise analysis revealed Surprise as the best-performing emotion with an F1-score of 97.9% and AUC of 0.989. The proposed model significantly outperforms existing approaches including ResNet-based hybrid models (97.3%), CNN-GRU architectures (96.5%), and standard CNN models (95.0%). Confusion matrix analysis indicates minimal misclassifications, with the highest confusion occurring between Fear and Surprise at 2%. Training accuracy increased from 81% to 98%, with validation accuracy reaching 97%, demonstrating effective learning and generalization. The findings confirm that integrating CNN and LSTM architectures with attention mechanisms significantly improves facial emotion recognition performance, making the proposed system suitable for practical applications in affective computing and intelligent human-computer interaction