Adaptive Ensemble Learning for Accurate Classification of High-Dimensional Data

Abstract

The proliferation of high-dimensional data in genomics, text analytics, hyperspectral imaging and industrial sensing has exposed a persistent weakness of conventional classifiers: as the number of features grows far beyond the number of available samples, distance measures lose contrast, decision boundaries become unstable, and models overfit noise rather than signal. This paper proposes an Adaptive Ensemble Learning (AEL) framework that addresses this small-n-large-p regime through three coupled mechanisms. First, relevance-biased stochastic subspace generation constructs diverse yet informative feature views using a composite mRMR-ReliefF ranking, so that base learners are neither confined to the same dominant features nor flooded with noise. Second, a heterogeneous pool of base learners is scored by a competence measure that jointly rewards out-of-bag accuracy, pairwise disagreement and prediction stability, after which redundant or weak members are removed by diversity-aware pruning. Third, ensemble weights are refined iteratively through a temperature-controlled softmax update rather than fixed at training time, allowing the ensemble to reallocate influence as competence estimates sharpen. The framework was evaluated on six benchmark high-dimensional datasets containing between 617 and 12,600 features. AEL attained a mean accuracy of 94.2 per cent, improving on the strongest baseline by 3.5 percentage points, and the gain widened as dimensionality increased. A Friedman test followed by Nemenyi post-hoc analysis confirmed that the improvement is statistically significant at the 0.05 level, while an ablation study showed that all three mechanisms contribute non-trivially to the final result.