Genomic prediction in small simulated populations using linear and nonlinear models

Abstract

This study compared the predictive performance of linear and nonlinear models using simulated genomic data under different broad-sense heritability scenarios. A forward-time stochastic simulation was conducted with 2,000 diploid individuals, and genomic prediction was performed using GBLUP, RR-BLUP, Deep Neural Networks, Support Vector Machines, Natural Gradient Boosting, and Extreme Gradient Boosting. The simulated genetic architecture included additive, dominance, and additive × additive epistatic effects. Models were trained using simulated pre-corrected phenotypic values and evaluated against simulated total genomic values. Model performance was assessed under three broad-sense heritability levels, H² = 0.10, 0.25, and 0.50, using prediction accuracy, mean absolute error, and mean squared error of prediction. GBLUP and RR-BLUP showed the highest prediction accuracies across the evaluated broad-sense heritability scenarios, indicating greater ability to rank individuals according to simulated total genomic merit. In contrast, nonlinear models, particularly Support Vector Machines, showed lower prediction errors, indicating greater numerical precision in estimating simulated total genomic values. These results suggest that linear and nonlinear models have complementary roles in genomic prediction, with linear models being more suitable for ranking individuals and nonlinear models contributing to reduced prediction error in the simulated scenarios evaluated. Keywords: Animal breeding; Genomic prediction; Machine learning; Simulated data.