Gaussian-mixture calculations and a medical diagnosis example illustrate how uncertainty-dependent labeling mechanisms can improve estimation and classification under a fixed labeling budget.
Abstract
Missing labels are usually regarded as a source of information loss in classification. We study a semi-supervised setting in which the probability of label missingness depends on the observed features through posterior classification uncertainty. In this setting, the missingness indicator is not only a record of an unobserved label, but also an observable signal generated by a mechanism linked to the classifier. We develop a likelihood-based information theory for such uncertainty-dependent missing labels. Under correct specification, we derive a Fisher-information decomposition that separates a partial-labeling component from a nonnegative mechanism-curvature term. Under joint misspecification of the label model and the missingness mechanism, we obtain the corresponding Godambe--Eicker--Huber--White sensitivity and sandwich-covariance partitions. We also clarify the relevant complete-data benchmark: favorable missingness can increase information relative to ordinary fully labeled or budget-matched non-informative labeling baselines, but cannot exceed the information in the augmented experiment in which labels and mechanism indicators are both observed. For plug-in classifiers, we connect the information decomposition to margin-based excess-risk bounds. In regular two-component mixture settings this yields the parametric \(n^{-1}\) excess-risk rate, with constants determined by the nuisance-adjusted information in discriminant directions. Gaussian-mixture calculations and a medical diagnosis example illustrate how uncertainty-dependent labeling mechanisms can improve estimation and classification under a fixed labeling budget.
Semi-supervised classifiers are commonly trained from samples in which all features are observed but some class labels are missing. When label missingness is independent of the observed data, unavailable class memberships reduce Fisher information relative to a completely classified sample. We study a different regime in which the probability of label missingness depends on posterior classification uncertainty, so that the observed missing-label indicators can themselves carry information about the Bayes decision boundary. Building on the conditionally weighted information decomposition of Ahfock and McLachlan, we develop this phenomenon for a two-component exponential mixture. Although the exponential model is non-Gaussian, asymmetric, and supported on the positive half-line, its log-posterior odds remain linear in the feature. We derive Bayes'rule and its exact error rate, formulate entropy-logistic and squared-discriminant missingness mechanisms, and obtain the full partially classified likelihood. We then derive a decomposition of the Fisher information into the complete-data information, the conditionally weighted loss due to missing labels, and the information contributed by the missing labels. Numerical quadrature identifies regions in which the full likelihood classifier has asymptotic relative efficiency above or below one. Monte Carlo experiments with finite training samples broadly support the population calculations, with the largest departures from the asymptotic predictions occurring near the transition at which the relative efficiency crosses one.
Huanchao Zhou, Jinran Wu, Fariborz Setoudehtazang et al.· 0 citations
Multi-Label Learning (MLL) refers to inducing multi-label prediction models from the precisely labeled training dataset. However, in many real-world scenarios, e.g., crowdsourcing annotations, the training datasets are often only partially valid, where each training instance is associated with a candidate label set, covering ground-truth labels but also with irrelevant ones. Naturally, learning with such datasets, formally referred to as Partial Multi-label Learning (PML), involves many noisy supervised signals, hence imposing a significant challenge to the prediction model induction. To meet this challenge, we purify the noisy supervised signals by formulating the latent label distribution, i.e., the probability of a candidate label being a ground-truth one, and then jointly learn it with the prediction model by minimizing their regularized Wasserstein distance, i.e., a robust distance for distributions as well as involving label correlations. Therefore, we propose a novel PML method, namely Wasserstein Partial Multi-Label Learning with dual Label Correlation Perspectives (Wpml3cp), solved by the gradient descent with an augmented Lagrange multiplier technique. To further enhance the robustness of Wpml3cp against exceptionally high ratios of irrelevant labels, we extend it with a Dual-branch Competitive Cleansing mechanism, leading to Wpml3cp-D. Besides, we also analyze the generalization error bound and time complexity of Wpml3cp and Wpml3cp-D. The extensive experiments are constructed by comparing Wpml3cp and Wpml3cp-D with existing PML baselines across synthetic and real-world datasets, and empirical results demonstrate that Wpml3cp and Wpml3cp-D can outperform the PML baselines in various noisy levels.
Ximing Li, Yuanchao Dai, Bing Wang et al.· ACM Transactions on Knowledg...· 0 citations
In many classification problems, reliable instance-level labels are unavailable. However, it is often possible to construct weakly enriched unlabeled samples: datasets selected by different cuts, sources, populations, or experimental conditions that change latent class proportions without revealing them. Classification without Labels (CWoLa) shows that, in the binary case ($K=2$), a classifier trained to distinguish two impure mixtures with different class proportions can recover an optimal class discriminator without knowing the mixture proportions. We extend this principle to multiclass learning from several unlabeled mixtures ($K>2$), where the learner observes only mixture identity and neither latent class labels nor class-prior matrices. We prove that, for a multiclass mixture model, the Bayes-optimal mixture classifier $g^\star$ maps data points into a $(K-1)$-simplex embedded in mixture-posterior space. The $K$ vertices of this simplex are induced by the latent classes through the unknown mixing matrix. Leveraging this geometry, we propose prior-free procedures that train a standard classifier to distinguish mixture identities and then extract latent class structure using either post-hoc simplex fitting or a bottleneck architecture. Experiments on MNIST, CIFAR-10, and Galaxy10 DECaLS show that mixture identity alone can recover latent classes and their fractions in the mixture. By narrowing the gap between weakly supervised and fully supervised performance, we provide a mathematically grounded, scalable tool for multiclass discovery in label-scarce domains.
Raphaël Bonnet-Guerrini, Johann Ioannou-Nikolaides, Troels C. Petersen et al.· 1 citation
C-Score, a compact framework that evaluates training behavior in three complementary spaces: prediction, feature representation, and optimization, suggests that clean accuracy alone is insufficient for evaluating SSL robustness in open-world environments, and that internal diagnostic signals are necessary for more reliable robustness assessment under unlabeled contamination.
Tsao-Lun Chen, Chicheng Fu, Han-Yi Chou et al.· 0 citations
: Partial multi-label learning addresses scenarios where each instance is associated with a set of candidate labels that include both relevant and irrelevant ones. In practical scenarios, such label sets are often simultaneously incomplete and noisy, which severely hampers the ability of models to extract compact and discriminative features. To address these issues, we propose an integrated learning paradigm that simultaneously enhances feature compactness and improves robustness against label noise. Our method learns an adaptive fuzzy neighborhood graph to capture the intrinsic relationships among instances. The resulting graph enables reliable label propagation, which effectively rectifies incorrect annotations and infers missing labels. In addition, we introduce a feature disentanglement mechanism that isolates reliable label-related feature representations from spurious ones introduced by noisy supervision. By integrating feature learning and label refinement into a joint optimization process, the proposed approach achieves a synergistic improvement in both representation quality and label reliability. Extensive theoretical analysis and empirical studies on multiple benchmark datasets demonstrate that our framework consistently outperforms state-of-the-art methods in terms of accuracy, stability, and robustness to annotation noise.
Yuzhi Tao, Anhui Tan· Computers, Materials & C...· 0 citations
Multi-label complementary label learning (MLCLL) is a weakly supervised paradigm that addresses multi-label learning (MLL) tasks using complementary labels (i.e., irrelevant labels) instead of relevant labels. Existing methods typically adopt an unbiased risk estimator (URE) under the assumption that complementary labels follow a uniform distribution. However, this assumption fails in real-world scenarios due to instance-specific annotation biases, making URE-based methods ineffective under such conditions. Furthermore, existing methods un-derutilize label correlations inherent in MLL. To address these limitations, we propose ComRank , a ranking loss framework for MLCLL, which encourages complementary labels to be ranked lower than non-complementary ones, thereby modeling pairwise label relationships. Theoretically, our surrogate loss ensures Bayes consistency under both uniform and biased cases. Experiments demonstrate the effectiveness of our method in MLCLL tasks. The code is available at https://github.com/JellyJamZhu/ComRank.
Jin Zhu, Yi Gao, Miao Xu et al.· Neural Information Processin...· 0 citations