High-dimensional expensive mixed-variable optimization presents a significant challenge due to the curse of dimensionality and variable heterogeneity. Existing methods often struggle with blind dimension reduction and inefficient search guidance. This paper proposes a novel framework: Ensemble Surrogate-assisted hybrid search with importance-aware Dimensional Splitting (ESDS). Unlike traditional blind splitting, ESDS dynamically allocates the dimensional budget to the most critical variables via data-driven sensitivity analysis. To balance global exploration and local exploitation, a hybrid search mechanism is introduced to the predicted best individual, which is generated based on an ensemble surrogate. It competes with the historical best individual to guide the initialization of the trust region-based Bayesian optimization. Furthermore, an adaptive heterogeneous ensemble is employed, aggregating dynamic weighted predictions to mitigate single-model fragility. Experiments on seven benchmarks against six state-of-the-art baselines demonstrate ESDS's superior convergence efficiency.
Junfeng Yi, Shuwei Zhu, Meiji Cui et al.· Proceedings of the Genetic a...· 0 citations
Deep neural networks can achieve very high accuracy, but their architectures are most often designed by hand and tuned for a single goal, such as accuracy. In practice, we often care about several goals at once, for example, accuracy, model size, and computational cost. Multi-objective neural architecture search (MONAS) can find a set of architectures that balance these goals, but the resulting models are usually very different from each other, which makes them hard to maintain and deploy as a family. In this work, we study how to make the diverse trade-off architectures to possess a regularity, so they can be better understood, maintained, and deployed with confidence. To this effort, we first run a standard multi-objective NAS to obtain a set of Pareto neural architectures, then analyze the common structural patterns that appear frequently among them. We then fix these common patterns and run a second MONAS in the reduced search space to obtain a set of regularized neural architectures. Across NAS-Bench-101, NAS-Bench-201, and an AG News text classification task, we show that this regularity-driven search can produce families of architectures that remain competitive in performance while being structurally simpler and having a regular pattern.
Ritam Guha, Shashank Raj, Kalyanmoy Deb· Proceedings of the Genetic a...· 0 citations
Interactive multi-criterion decision-making (iMCDM) procedures allow a humam decision-maker (DM) to iteratively adjusts preferences—via objective classification, reference points/directions, or weights—and solves a scalarized problem until a satisfactory Paretooptimal solution is obtained. However, benchmarking a iMCDM procedure remains challenging due to involvement of human DM in making decisions resulting in different solutions. To address this challenge, this paper proposes a stochastic machine learning-based decision-maker: pMachine-DM, which emulates human preference articulation using two stochastic ANNs. The first ANN classifies all objectives according to their desired improvement, relaxation, or satisfaction, while the second ANN predicts the associated bounding parameters. The proposed pMachine-DM is implemented with a specific iMCDM procedure - STEP method. The effectiveness of the approach is demonstrated on four benchmark and four engineering problems. The proposed pMachine-DM is generic and can be integrated with other iMCDM procedures, enabling systematic and reproducible benchmarking without involving human DMs.
Deepanshu Yadav, Kalyanmoy Deb· Proceedings of the Genetic a...· 1 citation
Interactive multi-criterion decision-making (iMCDM) procedures rely on a human decision-maker (DM) to iteratively provide preferences, so scalarized subproblems can move toward a preferred Pareto-optimal solution. This human-in-the-loop nature makes systematic benchmarking difficult, as preference information and interaction patterns vary across individuals and problems. To overcome this limitation and to enable computationally-oriented researchers to contribute more profoundly in the MCDM field, we introduce a Machine-based Decision Maker (Machine-DM) that replaces human DMs with pre-trained machine learning models capable of performing the key iMCDM tasks automatically. The Machine-DM predicts objective classifications, such as which objectives should be improved, relaxed, fixed, or allowed to vary and generates corresponding bounding parameters without requiring knowledge of the true target preferred solution. Using this Machine-DM, we develop machine-based versions of four well-known iMCDM procedures: STEM, GUESS, STOM, and NIMBUS. We further propose a set of performance metrics designed to evaluate performance of these iMCDM procedures. Using the Machine-DM concept we also propose a Bench-iMCDM framework for benchmarking iMCDM procedures. Applications to test and engineering problems demonstrate the usefulness of Machine-DM and highlight its potential to serve as a unified framework for comparing a broad class of iMCDM procedures and also to develop new ones.
Deepanshu Yadav, Kalyanmoy Deb· Annual Conference on Genetic...· 2 citations
The method combines an interpretable cost-function formulation with a multi-objective evolutionary algorithm to discover Pareto-optimal ensemble configurations and introduces a two-stage combiner that produces diverse candidate responses and selects final outputs via embedding-based semantic consensus.