When Data Imbalance Helps: Robust Generalization Through Shortcut Saturation

When Clean Data Hurts: Learning with Monotone Corruptions Beyond Binary Classification

This work exhibits a learnable multiclass problem that becomes altogether unlearnable under a monotone adversary, and shows an analogous result for partial binary concept classes, and demonstrates that monotone adversaries are frighteningly more powerful in each of these settings.

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of the method. We show this practice is unsafe. On the public Kaggle credit-card fraud dataset, under a leakage-free nested cross-validation protocol in which the decision threshold is selected on a held-out inner validation fold, a plain Random Forest at the default 0.5 threshold attains F1 = 0.861 +/- 0.021, and threshold tuning yields it no benefit (delta-F1 = -0.002). Read alone, this supports an appealing conclusion: for a well-calibrated ensemble, imbalance handling is unnecessary. We then apply the identical protocol to 45 binary tasks spanning imbalance ratios from 1:1.5 to 1:178 (2,025 model fits, four model families). The conclusion reverses. Random Forest benefits most from threshold tuning across the suite (delta-F1 = +0.101 +/- 0.134), not least, while three other families replicate their fraud-dataset behaviour almost exactly. SMOTE likewise harms the fraud dataset but helps across the suite (mean delta-F1 = +0.076; 138 wins, 39 losses; Wilcoxon p = 2.7e-17). Two further results. Threshold-tuning benefit is non-monotonic in the imbalance ratio: near zero below 1:5, peaking at +0.120 in the 1:15-1:40 band, declining to +0.045 beyond 1:100 - explaining why the fraud dataset, at 1:577, is an unrepresentative place to study the question. And we reject an intuitive heuristic: validation-set calibration error does not predict tuning benefit (expected calibration error r = -0.087; Brier r = +0.137), so calibration diagnostics cannot tell a practitioner whether tuning is worthwhile. We release the protocol, the 45-task harness, and all per-run metrics.

Gradient Mirage: Trainable yet Label-Unidentifiable Gradients in Large Language Model Split Learning

Gradient matching attacks (GMAs) in LLM split learning (SL) rely on a critical yet underexplored assumption: the gradient exposed at the split interface is a faithful derivative of the client's full-label training objective. This gradient-objective consistency allows a curious server to recover private labels by searching for a sequence whose induced gradient explains the observation. We propose Gradient Mirage, a defense that breaks this consistency without discarding the optimization utility of the backward signal. Our key idea is to induce the adversary to solve a misspecified inverse problem, in which no plausible label sequence in the sequence space can explain the observed gradients. Concretely, Gradient Mirage achieves this by inducing inconsistency across three dimensions: objective, direction, and scale. Selective Autoregressive Supervision derives the exposed gradient from a masked surrogate loss rather than the full-label objective assumed by the attacker; Scale Blinding then applies randomized multiplicative rescaling, obscuring the gradient's natural magnitude; and Directional Privatization further randomizes the gradient direction while preserving its magnitude through the von Mises-Fisher (vMF) mechanism under a directional metric differential privacy guarantee. Crucially, utility is preserved: the Top segment still learns from all target tokens via Dual-Track Backpropagation, the exposed gradient remains informative since each supervised token retains its complete autoregressive context, and Bottom-Gradient Recovery restores the effective gradient for Bottom-segment optimization. Extensive experiments show that Gradient Mirage provides substantially stronger protection than existing defenses under comparable fine-tuning performance, achieving a better privacy-utility trade-off.

Bagging Robustly Learns VC Classes with Linear Sample Complexity

It is proved that VC classes are adversarially robustly learnable with sample complexity linear in the VC dimension $d$, providing an exponential improvement over the previous upper bound of Montasser, Hanneke, and Srebro (2019).

NADOS: Reliability-Gated Difficulty-Aware Oversampling for Noisy Imbalanced Classification

Class imbalance becomes more challenging when minority underrepresentation is accompanied by label noise, because synthetic oversampling may amplify unreliable local structures if noisy minority samples are used as interpolation seeds. This study proposes Noise-Aware Adaptive-Difficulty Oversampling (NADOS), which separates the assessment of seed trustworthiness from the allocation of synthesis effort. NADOS evaluates each minority instance through two local criteria. A reliability score determines whether the instance is suitable for synthetic generation, while a difficulty score assigns synthesis priority among eligible instances. The method is evaluated on 22 binary imbalanced benchmark datasets, five controlled label-noise levels, four classifier families, and nine oversampling methods. Across the full benchmark, NADOS obtains average F1 = 0.7428, G-mean = 0.8511, balanced accuracy = 0.8552, and AUPRC = 0.7700. These results give NADOS the strongest average F1, G-mean, and balanced accuracy, while remaining competitive on AUPRC. Non-parametric statistical tests show significant differences among the compared methods and indicate that NADOS is consistently competitive under noisy imbalanced learning conditions. Separating seed reliability from synthesis difficulty provides a practical strategy for noisy imbalanced classification. The reliability gate limits unsafe seed usage, while the difficulty score preserves attention to difficult but trustworthy minority samples.

GAN-powered oversampling with strengthened sample distribution for class overlapping imbalanced data