Jul 2026· ACM Transactions on Evolutionary Learning and Optimization· 0 citations· 67 references
TL;DR
This work proposes to reduce this expense by predicting the lower level Pareto set for a candidate upper level solution directly, in lieu of performing optimization from scratch, which is competitive across a range of problems, including both deceptive and non-deceptive problems.
Abstract
Bilevel optimization problems comprise an upper level optimization task with a lower level optimization task as a constraint. While there is a significant and growing literature devoted to solving bilevel problems with single objective at both levels using evolutionary computation, relatively little work has been reported on addressing problems with multiple objectives (BLMOP) at both levels. For highly non-linear or black-box BLMOPs, the existing evolutionary techniques typically employ nested search, which in its native form consumes large number of function evaluations. In this work, we propose to reduce this expense by predicting the lower level Pareto set for a candidate upper level solution directly, in lieu of performing optimization from scratch. Such prediction is significantly challenging for BLMOPs as it involves one-to-many mapping. To address this challenge, we supplement the dataset using a helper variable and construct a neural network, which can then be trained to map the variables in a meaningful manner. Then, we embed this initialization within a bilevel optimization framework, termed Pareto set prediction assisted evolutionary bilevel multi-objective optimization (PSP-BLEMO). Systematic experiments with existing state-of-the-art methods are presented to demonstrate its benefit in terms of solution quality and computational expense. The experiments show that the proposed approach is competitive across a range of problems, including both deceptive and non-deceptive problems.
Multi-objective optimization problems (MOPs) are prevalent in numerous real-world applications. Recently, Pareto Set Learning (PSL) has emerged as a powerful paradigm for solving MOPs. PSL can produce a neural network for modeling the set of all Pareto optimal solutions. However, applying PSL to black-box objectives, particularly those exhibiting non-separability, high dimensionality, and/or other complex properties, remains very challenging. To address this issue, we propose leveraging evolution strategies (ESs), a class of specialized black-box optimization algorithms, within the PSL paradigm. Traditional ESs capture the complex dimensional dependencies less efficiently, which can significantly hinder their performance in PSL. To tackle this issue, we suggest encapsulating the dependencies within a neural network, which is then trained using a novel gradient estimation method. The proposed method, termed Neural-ES, is evaluated using a bespoke benchmark suite for black-box PSL. Experimental comparisons with other methods demonstrate the efficiency of Neural-ES, underscoring its ability to learn the Pareto sets of challenging black-box MOPs.
Chengyu Lu, Zhenhua Li, Xi Lin et al.· Neural Information Processin...· 0 citations
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
LLM-driven evolutionary search can discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design, with SMAC hyperparameter optimization integrated into the evolutionary loop.
G. Laskaris, R. Brasher, Niki van Stein et al.· 0 citations
This work presents a surrogate-assisted multi-objective optimization approach that leverages symbolic regression (SR) to construct interpretable analytical approximations of expensive black-box functions. It uses iterative construction of surrogates (symbolic models) and Tchebycheff scalarization along a set of parallel reference vectors to search for well-distributed solutions on the Pareto front. Optimization of the symbolic models is performed using a nonlinear programming (NLP) solver in lieu of heuristic search. A distance-based subset selection strategy is used to select a candidate for true evaluation, ensuring efficient use of limited evaluation budget in a steady-state framework. The approach is compared against a Kriging-assisted NSGA-II, also implemented within a steady-state framework. Experiments are performed on six benchmark problems—covering both constrained and unconstrained cases, including a practical engineering benchmark of bracket design problem. The findings highlight the strengths and limitations of SR-based surrogates combined with NLP, and position them as a viable alternative to conventional surrogate-assisted evolutionary algorithms.
Kannan Sekar, H. Singh, Tapabrata Ray· Proceedings of the Genetic a...· 0 citations
Large-scale sparse multiobjective optimization problems (LSSMOPs) involve a large number of decision variables and Pareto optimal solutions with only a few nonzero variables. However, as the number of decision variables grows, it becomes increasingly challenging to accurately identify the nonzero variables, and optimization performance is adversely affected. To address these issues, this paper proposes an evolutionary algorithm for LSSMOPs. Specifically, we propose a new initialization method capable of generating scores that accurately reflect the importance of variables, and an initial mask vector template that can locate nonzero variables. This leads to the generation of a high-quality initial population. Additionally, this paper introduces a new strategy to calculate the mutation probability for each variable and a novel optimization for real variables based on the Pareto-guided normal distribution, enabling the population to avoid being trapped in local optima and quickly converge to the global optimum. Experimental results from eight benchmark problems and three real-world applications demonstrate that the proposed algorithm achieves superior performance compared with state-of-the-art algorithms.
Jia-Lin Mai, Min-Rong Chen, Guoqiang Zeng et al.· 0 citations
Comprehensive experiments on numerical benchmarks and neuroevolution robotic control tasks demonstrate that AutoPSO consistently discovers novel PSO variants that significantly outperform strong baselines and confirm that AutoPSO achieves increasing performance gains with larger swarm sizes.
Xin-Meng Yu, Jiaxin Gao, Jianguo Zhang et al.· IEEE Transactions on Evoluti...· 0 citations