Attention-enhanced LSTM framework for stock forecasting using Bayesian optimization

Abstract

Background/Objectives: Highly non-linear financial data makes accurate stock market forecasting challenging. While traditional Long Short-Term Memory networks excel at capturing sequential trends, they often struggle with long-range dependencies of complex market. Addressing these limitations, this study aims to propose a hybrid framework by integrating Multi-Head Self-Attention (MHSA) mechanism into baseline LSTM architecture to dynamically weight critical historical features, combined with Bayesian optimization to systematically refine hyper-parameters for superior forecasting accuracy. Method: The architectural upgrade of integrating robust attention layer allows to compute contextual weights, focusing on critical long-term temporal dependencies within financial data. To maximize forecasting accuracy, Bayesian optimization is systematically applied over entire hybrid structure. This automated process evaluates complex hyper-parameter space, efficiently identifying ideal combination to minimize prediction error. Findings: Empirical results demonstrate that attention-enhanced LSTM framework significantly outperforms baseline model in forecasting accuracy. By dynamically weighting temporal features, MHSA mechanism reduced MSE by 14.5% and MAPE to 1.82%, down from LSTM's 2.45%. Furthermore, Bayesian optimization efficiently converged on optimal hyper-parameters within fewer iterations (65% reduction in tuning time), eliminating manual tuning bias. This optimization combined with architectural enhancement allowed the model to achieve improved directional accuracy, robustly capturing market volatility, and proving model efficacy for complex financial forecasting. Novelty: The proposed framework advances financial forecasting by embedding self-attention into baseline LSTM, uniquely coupled with automated Bayesian optimization. This integration enables to dynamically capture volatile market trends with unprecedented mathematical precision, dramatically reducing prediction error and hyper-parameter tuning time. Keywords: Stock market prediction, Bayesian optimization, Long short-term memory, Attention mechanism, Feature enhancement