Evaluating the performance of Greedy K-center across a variety of metric spaces shows that mapping unlabeled instances into a predictive probability space and weighting the result by entropy often dominates the other options for active learning selection with Greedy K-center.
Abstract
With rapid advancement over the last few years, many different methods are now widely used for classification. However, training these models requires substantial labeled data. Active Learning is a potential solution to this problem. Pool-based active learning minimizes costs by querying only the most informative samples from an unlabeled dataset. Diversity-based approaches, on the other hand, attempt to select a representative subset of the data. There are many different objectives for determining the selection process, including exact K-center, exact K-median, and Greedy K-center. In this paper, we will focus on evaluating the performance of Greedy K-center across a variety of metric spaces: the raw feature space, a Linear Discriminant Analysis (LDA) space, and a model-derived probability space (with and without entropy-based weighting). Using Random Forest classifiers as a baseline evaluator, our empirical results on synthetic and real-world datasets demonstrate that mapping unlabeled instances into a predictive probability space and weighting the result by entropy often dominates the other options for active learning selection with Greedy K-center.
It is demonstrated that matrix multiplication can be reformulated as a series of convolutions by exploiting a special structure in the matrix using the Fast Fourier Trans-form (FFT), reducing the time complexity to O ( q 2 log q ) .
Imbalanced multiclass learning remains challenging due to skewed class distributions, class overlap, and heterogeneous within-class structure. We revisit the Multiclass Imbalance Learning in Ensembles through Selective Sampling (MILES) framework and study two clustering-based variants: MILES \({}^{k}\) , which uses \(k\) -means with an SSE-based heuristic for selecting the number of clusters, and MILES \({}^{FF}\) , which uses FarthestFirst to explore an alternative centroid-based partitioning strategy. Both variants combine clustering-guided selective sampling with resampling to construct diverse and more balanced training subsets, improving representation of difficult classes while preserving local decision structure. We evaluate MILES on eight real-world multiclass datasets spanning different imbalance regimes, overlap levels, and feature complexities. Across these experiments, MILES is consistently competitive in Multiclass Area Under the Curve ( \(MAUC\) ) and achieves its strongest gains in Macro-F1, with statistically significant improvements over several strong ensemble baselines. MILES also achieves strong Micro-F1 on multiple datasets, while its performance on Geometric Mean ( \(G\) -Mean) is competitive but more dataset-dependent. Class-wise analysis further shows that the improvement arises from better recovery of hard classes and systematic reduction of dominant baseline confusion patterns, especially on Page and Satellite, while Hyperspectral highlights a limitation case where high dimensionality and stronger overlap reduce the benefits of centroid-based selective sampling. A glaciology case study on glacier algae prediction further demonstrates the practical utility of the framework. Overall, the results position MILES as a robust and generalizable alternative to conventional boosting, bagging, and cost-sensitive ensembles for multiclass imbalanced learning.
Bayu Adhi Tama, A. Azari, V. Janeja· ACM Transactions on Intellig...· 0 citations
The proliferation of high-dimensional data in genomics, text analytics, hyperspectral imaging and industrial sensing has exposed a persistent weakness of conventional classifiers: as the number of features grows far beyond the number of available samples, distance measures lose contrast, decision boundaries become unstable, and models overfit noise rather than signal. This paper proposes an Adaptive Ensemble Learning (AEL) framework that addresses this small-n-large-p regime through three coupled mechanisms. First, relevance-biased stochastic subspace generation constructs diverse yet informative feature views using a composite mRMR-ReliefF ranking, so that base learners are neither confined to the same dominant features nor flooded with noise. Second, a heterogeneous pool of base learners is scored by a competence measure that jointly rewards out-of-bag accuracy, pairwise disagreement and prediction stability, after which redundant or weak members are removed by diversity-aware pruning. Third, ensemble weights are refined iteratively through a temperature-controlled softmax update rather than fixed at training time, allowing the ensemble to reallocate influence as competence estimates sharpen. The framework was evaluated on six benchmark high-dimensional datasets containing between 617 and 12,600 features. AEL attained a mean accuracy of 94.2 per cent, improving on the strongest baseline by 3.5 percentage points, and the gain widened as dimensionality increased. A Friedman test followed by Nemenyi post-hoc analysis confirmed that the improvement is statistically significant at the 0.05 level, while an ablation study showed that all three mechanisms contribute non-trivially to the final result.
Porwal Rabins· International Journal of Inn...· 0 citations
Mining classification rules for the minority class is challenging not only because positive examples are rare, but because they concentrate in small, geometrically irregular subregions of feature space that unguided search methods systematically miss. We propose a clustering-based knowledge base that captures positive-class structure before search begins and uses it to focus both initialization and neighborhood exploration in Moca-I, a multi-objective local-search algorithm that mines interpretable rule sets by trading off minority-class recall, precision, and complexity. Rather than sampling attribute conditions blindly from the full discretized space, the knowledge base clusters positive-class instances, extracts per-cluster attribute ranges, and aligns them with the algorithm's discretization—seeding the initial archive with minority-class-informed prototypes and restricting neighborhood operators to locally relevant regions. We instantiate this framework with two clustering methods: Self-Organizing Maps (MOCA-ISOM), which preserve the topological structure of the positive-class manifold, and K-Means (MOCA-IKM), a centroid-based baseline. Evaluated on 19 imbalanced benchmark datasets, MOCA-ISOM achieves statistically significant F-measure improvements on eight datasets and Hypervolume improvements on nine, with gains up to +17% absolute on high-dimensional data. MOCA-IKM shows comparable average performance but exhibits five statistically significant degradations.
Raymonde Akiki, Chantal Saad Hajjar, M. Chamoun et al.· Proceedings of the Genetic a...· 0 citations
In multiple instance regression (MIR) data are organized into bags (collections of instances in feature space) and the goal is to learn a mapping that assigns labels to bags. A typical assumption is that there is a so-called concept point in feature space, the proximity to which dictates the bag label. Motivated by modern MIR architectures which are based on attention, we study a softmax model that decouples the concept point and the labeling scheme. The two are respectively determined by a query direction and a value direction value in feature space. This problem isolates a basic challenge of learning both the query and value vectors from bag-level supervision. From this model we derive a parametric family of iterations in the noiseless limit, which generalizes a method known as the EM-DD algorithm. We then derive concentration results for the MLE estimators of the query and value vectors obtained from a random selection of instances. Our result for the value vector shows that a single random initialization of the value vector already points in the correct direction on average, so that a polynomial (in the number of instances per bag and the feature dimension) number of bags is enough for the EM algorithm to converge in $O(1)$ steps with high probability. A key aspect of this analysis is the interplay between concentration of empirical covariance matrices and extremal statistics arising from the selection rule.
The rapid growth of image data has produced large-scale datasets, raising concerns about the time and memory costs of model training. Selecting representative training subsets, however, remains challenging: individual sample contributions are unclear, and model behavior varies across datasets and runs. We address these challenges with a framework that combines coreset selection with an ensemble aggregation over multiple runs. For coreset selection, we propose SCOre-Stratified Selection (SCOSS), which partitions the training data into intervals based on a chosen score and samples from each interval. The ensemble combines predictions from multiple runs, each performed on an independently sampled training subset. As baselines, we use moderate and random selection, each in original and class-balanced versions. We assess the framework with Simple Graph Convolution (SGC) and Support Vector Machine (SVM) classifiers under different sampling ratios. Experiments show that SCOSS is competitive with baselines, often the best choice for SGC, and enables favorable trade-offs between accuracy and efficiency. On the fine-grained dataset, SGC with SCOSS outperforms SVMs when using fewer labeled samples. The code and supplementary materials are publicly available at http://scoss.lucasvalem.com.
