Parallel Optimized Particle Swarm Optimization: An Efficient Parallel Approach for Neural Network Training

Abstract

Global optimization has become a cornerstone of modern machine learning, particularly for the challenging task of training high-dimensional neural network architectures. Despite the prevalence of established metaheuristics, traditional approaches frequently struggle with the irregular and multimodal error surfaces inherent in complex training landscapes. This paper proposes the parallel optimized particle swarm optimization (Parallel OPSO), a multi-population parallel framework specifically engineered to enhance the training efficiency and predictive accuracy of neural networks across classification and regression tasks. At its core, OPSO integrates several advanced mechanisms: a simulated annealing-based acceptance criterion to escape local minima, adaptive inertia weights that dynamically respond to stagnation by injecting momentum, and a hybrid Broyden–Fletcher–Goldfarb–Shanno (BFGS) local search phase for fine-grained convergence. While OPSO defines this underlying algorithmic model, Parallel OPSO represents its execution across parallel multi-population architectures. By partitioning the swarm into interacting subpopulations and leveraging modern multi-core architectures, Parallel OPSO significantly reduces training time, with prominent performance gains primarily manifesting on highly challenging and multimodal optimization tasks. Experimental evaluations demonstrate that while standard algorithms may perform adequately on simpler loss landscapes, Parallel OPSO achieves superior convergence robustness and accuracy when navigating demanding and highly irregular error surfaces. These results underscore Parallel OPSO as a highly efficient and scalable framework for optimizing neural network parameters in complex search spaces.