Search Trajectory Networks (STNs) are a graph-based tool for visualizing and characterizing the behavior of optimization algorithms. STNs'reliance on discretization of the search space has largely confined them to low-dimensional or combinatorial settings. We introduce a methodology for constructing STNs in semantic spaces, defined as the space of a model's predictions on a fixed sample set. Our approach discretizes semantic vectors and aggregates them into network nodes via agglomerative clustering with complete linkage under a normalized Hamming distance. Since any predictor can be summarized by its semantic vector, this method enables comparison of learning dynamics across otherwise incomparable algorithm families. We apply semantic space STNs to classification and regression tasks solved using different machine learning algorithms, recovering known qualitative differences between them. Additionally, we use semantic space STNs to study neural network generalization by contrasting standard training with the label randomization regime of Zhang et al. (2017). The resulting STNs exhibit consistent structural differences, training on real labels produces denser, more efficient and more centralized graphs than training on shuffled labels. Together, our results show that semantic space STNs capture functional training dynamics arising from the interaction between learning algorithms and data, providing a tool for analyzing and comparing learning dynamics across machine learning models and training regimes.
J. Agudelo, A. Tonda, Gabriela Ochoa et al.· 0 citations
Fitness landscape analysis provides insights into optimization problems, informing algorithms' design and identifying properties that influence performance. While understanding global landscape structure is critical, tools for analyzing and visualizing multi-objective, high-dimensional optimization problems remain limited. Recent models, such as Pareto local optima solution networks (PLOS-nets), primarily focus on small instances and binary representations, posing challenges for extension to more complex domains. To address this gap, we introduce mo-LON/D, a decomposition-based local optima network model for multi-objective landscapes. This model partitions a multi-objective problem into scalar sub-problems, constructs standard single-objective local optima networks (LONs) for each, and integrates them via a graph union. We validate mo-LON/D on fully enumerated bi-objective ρmnk-landscapes and contrast its structural features against PLOS-nets both visually and quantitatively. Despite the inherent sampling involved in scalarization, our results indicate that mo-LON/D offers comparable explanatory power (and even higher correlations) with respect to the performance of state-of-the-art algorithms. By harnessing established sampling techniques from single-objective research, mo-LON/D could potentially provide a scalable framework for characterizing complex multi-objective landscapes.
Gabriela Ochoa, Quentin Renau, Arnaud Liefooghe et al.· Annual Conference on Genetic...· 0 citations