Overlap Mitigation Versus Classifier Selection in Imbalanced Classification: A Dual-Baseline Analysis
In imbalanced binary classification, class overlap forces a familiar fork in the road: refine a preprocessing pipeline, or pivot to classifier selection. Which path dominates has remained empirically open. To address this question, a full factorial experiment was run across 58 KEEL datasets, 11 mitigation techniques, and 10 classifiers (yielding 6960 fitted models in total), with results read through a dual-baseline lens. Against the same classifier without mitigation, the techniques delivered a mean improvement of +4.13 points; gains concentrated on weak learners, with SMOTE and BorderlineSMOTE topping the rankings. Against an honest leave-one-dataset-out (LODO) baseline that selects the strongest standalone classifier from a held-out training pool, mean ΔF1 fell to −4.64 points, and 43.2% of mitigated configurations matched or beat the baseline. Among controllable factors, a Type II ANOVA placed classifier identity at η2 = 0.157 against mitigation technique at η2 = 0.010, so the choice of algorithm carried roughly sixteen times the explanatory weight of the choice of preprocessing. Dataset characteristics dominated the overall variance at η2 = 0.620. The honest LODO selector returned LightGBM on every one of the 58 datasets, suggesting that under typical practitioner conditions, a strong-classifier default is the lower-effort, lower-variance path, with no expected performance cost relative to a pipeline-development investment.