Mixed non linear programming model for optimizing and training long short-term memory architectures: application in traffic flow forecasting

Deep Learning for VIX Volatility Forecasting: A Comparative Study of LSTM, BiLSTM, and CNN-LSTM Architectures

This study evaluates the predictive performance of three deep learning architectures: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and a hybrid Convolutional Neural Network-LSTM (CNN-LSTM), for forecasting the CBOE Volatility Index (VIX). Using daily VIX data from January 2000 to June 2024, we benchmark these models using mean absolute error (MAE), mean squared error (MSE), and the coefficient of determination (R2). The CNN-LSTM model consistently outperforms both unidirectional and bidirectional LSTMs, achieving the lowest MAE (1.758), MSE (9.033), and highest R2 (0.872). Contrary to expectations, the BiLSTM performs worst among the three, with an R2 of 0.670, and this suggests that bidirectional information flow may introduce noise rather than improve accuracy for volatility forecasting. The results also indicate that deep learning models maintain predictive stability during periods of market turbulence, with CNN-LSTM demonstrating the strongest resilience. These findings have practical implications for real-time risk monitoring systems in volatile financial markets.

GA-optimized BiLSTM with dynamic surrogate model for enhanced stock price prediction

Machine Learning Versus Deep Learning: SVR and LSTM Models for WTI Price Forecasting

This study compares Support Vector Regression (SVR) and Long Short-Term Memory (LSTM) models for forecasting annual West Texas Intermediate (WTI) crude oil prices using data from 1981-2023, with projections to 2035. The US Dollar Index (DXY) is incorporated as an explanatory variable to capture exchange-rate effects in global oil markets. A walk-forward crossvalidation framework is employed, and forecasting performance is evaluated using MSE, RMSE, MAE, MAPE, and $\mathrm{R}^{2}$. Results reveal a moderate negative correlation between WTI prices and the DXY index. Forecast comparison tests, including the paired t-test, Wilcoxon signed-rank test, and Diebold-Mariano (DM) test, consistently show that SVR outperforms LSTM. Incorporating DXY further improves forecasting accuracy, particularly for SVR. The extended SVR model achieves the highest explanatory power $\left(\mathrm{R}^{2}=0.928\right)$, compared with the baseline SVR $\left(\mathrm{R}^{2}=0.912\right)$, baseline LSTM $\left(\mathrm{R}^{2}=0.726\right)$, and extended LSTM $\left(\mathrm{R}^{2}=0.781\right)$. These findings suggest that SVR augmented with macro-financial information provides a more suitable framework for medium-term energy and fiscal policy analysis.

Comparative Study of LSTM, GRU, and CNN-Based Hybrid Models for Short-Term Load Forecasting

This study investigates the effectiveness of long short-term memory (LSTM) networks, Gated Recurrent Unit Networks (GRU), and their hybrid models with a Convolutional Neural Network (CNN), namely CNN-GRU and CNN-LSTM, in the short-term electrical load forecasting of practical power systems. This study uses hourly load data spanning 11 years from 2014 to 2024 from a typical regional transmission network. A set of experiments was conducted using various input configurations, model complexities, and activation functions. The models were tested in univariate and multivariate settings, including ten input features and recurrent layer variations. The results indicate that GRU consistently achieves the best performance in terms of mean absolute percentage error, whereas CNN-RNN hybrid models show sensitivity to the depth of architecture and choice of activation function. These deep learning methods were compared in terms of their model complexity and forecasting accuracy to provide guidance for selecting the optimal models for short-term electrical load forecasting. 

NeuroFlexMLP: A Low Complexity MLP Architecture for Long-Term Time Series Forecasting

Forecasting time series over long horizons is essential for proactive decision-making in many systems. Recent research has focused on transformer-based architectures, which capture long-range dependencies in sequential data. However, several studies show that simpler linear models can outperform transformers by avoiding overfitting during training. In this context, we present NeuroFlexMLP, a deep learning model for multivariate time series forecasting tasks. NeuroFlexMLP's key distinct feature is the adaptability to the diverse complexity of real-world time series, which is achieved, from the architecture standpoint, by adding non-linear residual blocks to a first linear block. This architectural design simplifies hyperparameter optimization, leading to accurate forecasts for various time series data types regardless of the lookback or prediction horizons, outperforming state-of-the-art (SOTA) models on challenging real-world datasets. Its Multi-Layer Perceptron (MLP) design ensures high computational efficiency, making it scalable for longer input sequences than transformer-based models. We validate NeuroFlexMLP for the LEO satellite beam hopping use case, where its lightweight design enables on-board deployment, and on state-of-the art AI datasets. Across all these benchmarks, NeuroFlexMLP achieves competitive accuracy over state-of-the-art models while providing an adaptive architecture that significantly reduces computational overhead. On the LEO beam hopping task, it achieves up to 35.9% MSE reduction over Informer, which translates into up to 28% lower provisioning cost under asymmetric cost models that penalize under-allocation more heavily than over-allocation.

Attention-enhanced LSTM framework for stock forecasting using Bayesian optimization

Background/Objectives: Highly non-linear financial data makes accurate stock market forecasting challenging. While traditional Long Short-Term Memory networks excel at capturing sequential trends, they often struggle with long-range dependencies of complex market. Addressing these limitations, this study aims to propose a hybrid framework by integrating Multi-Head Self-Attention (MHSA) mechanism into baseline LSTM architecture to dynamically weight critical historical features, combined with Bayesian optimization to systematically refine hyper-parameters for superior forecasting accuracy. Method: The architectural upgrade of integrating robust attention layer allows to compute contextual weights, focusing on critical long-term temporal dependencies within financial data. To maximize forecasting accuracy, Bayesian optimization is systematically applied over entire hybrid structure. This automated process evaluates complex hyper-parameter space, efficiently identifying ideal combination to minimize prediction error. Findings: Empirical results demonstrate that attention-enhanced LSTM framework significantly outperforms baseline model in forecasting accuracy. By dynamically weighting temporal features, MHSA mechanism reduced MSE by 14.5% and MAPE to 1.82%, down from LSTM's 2.45%. Furthermore, Bayesian optimization efficiently converged on optimal hyper-parameters within fewer iterations (65% reduction in tuning time), eliminating manual tuning bias. This optimization combined with architectural enhancement allowed the model to achieve improved directional accuracy, robustly capturing market volatility, and proving model efficacy for complex financial forecasting. Novelty: The proposed framework advances financial forecasting by embedding self-attention into baseline LSTM, uniquely coupled with automated Bayesian optimization. This integration enables to dynamically capture volatile market trends with unprecedented mathematical precision, dramatically reducing prediction error and hyper-parameter tuning time. Keywords: Stock market prediction, Bayesian optimization, Long short-term memory, Attention mechanism, Feature enhancement