Exploration in sparse-reward long-horizon tasks poses significant challenges for reinforcement learning. To address these challenges, we propose a two-level Hierarchical Reinforcement Learning (HRL) framework. The first level handles high-level strategic planning, while the low-level uses the continuous-control Soft Actor-Critic (SAC) algorithm, and they utilize entropy-regularized policy optimization. The proposed framework was trained and evaluated using the Search-and-Rescue-2 (SAR-2) dataset. HRL-SAC effectively addresses sparse-reward long-horizon search problems characterized by delayed rewards and continuous control, and its outperforming the flat SAC baseline reinforcement learning in terms of success rates, coverage efficiency, and convergence. These findings indicate that hierarchical entropy-regularized policies are a promising solution to tackle long-horizon sparse-reward reinforcement learning tasks.
Zahra Abdalla Elashaal, Afef Hfaiedh, N. Khraief et al.· 0 citations
The reliable and efficient operation of power systems hinges on accurate electrical load forecasts over various horizons and objectives, ranging from long‐term grid adequacy estimation to day‐ahead system scheduling and real‐time dispatching. In this context, deep learning (DL) models outperform conventional statistical approaches at the expense of interpretability, making the research on interpretable algorithms essential. Among these methods, the temporal fusion transformer (TFT) has proven its validity across multiple sectors, while its applicability to short‐term load forecasting (STLF) lacks sufficient corroboration. This study evaluates the TFT within a multivariate STLF task concerning a department building of an Italian university, focusing on interpretability. The performance of the model is evaluated against established benchmarks: The TFT outscores all competitors, yielding a mean absolute percentage error of 7.40% on the test subset, 25% lower than the competitors. Jointly, the interpretability of the model is studied: Variable selection weights identify the most important features, a comprehensive investigation of attention patterns highlights the most critical timesteps for generating forecasts, and the Comaniciu distance helps detect regime shifts and significant events. Overall, this study presents an effective TFT‐based model for day‐ahead STLF in terms of interpretability and forecasting performance.
Alessandro Nicola, Giorgia Ghione, V. Randazzo et al.· Advanced Intelligent Systems· 0 citations
Small-data inverse design is challenging in engineering informatics when observations are heterogeneous, mixed-type, and constrained by physical relations among design variables. This work proposes a topology-aware surrogate framework guided by an Incremental Transformer (INCRT) for physics-constrained inverse design, applied to geopolymer mixture design. The method integrates intrinsic-dimensionality analysis, mixed-variable design-space representation, tabular surrogate prediction, INCRT-based manifold rationalisation, and constrained inverse optimisation. Using a public benchmark of fly-ash and slag-based geopolymer concrete mixtures with compressive-strength and carbon-emission targets, the high-dimensional design space proves strongly redundant, organising around fewer effective mixture regimes. Compressive strength requires nonlinear tabular surrogates, while carbon emission is largely determined by composition and well recovered by regularised linear models. INCRT thus acts not as a replacement for tabular predictors but as a rationalisation layer providing prototype regimes and a manifold-support score for inverse design. Three strategies are compared: unconstrained surrogate optimisation, physics-constrained optimisation, and topology-aware physics-constrained optimisation. Unconstrained optimisation can match target strength but may yield physically invalid or off-manifold candidates; physics-only constraints do not always ensure data support. The topology-aware strategy yields candidates balancing target compliance, carbon reduction, physical admissibility, and proximity to the learned feasible manifold. The framework aims not to replace experimental validation but to support screening of credible candidate mixtures from small, mixed, physically constrained engineering datasets.
