The results show that injection-driven, weakly supervised training can enable scalable and consistent Real-Bogus classification without human-labeled training data while providing calibrated uncertainties, and achieves competitive calibration relative to more expensive ensemble baselines.
Abstract
Time-domain surveys generate many transient candidates, making Real-Bogus classification a critical step in automated discovery pipelines. Reliable labels are costly, while community labels can be noisy and survey-dependent. We aim to develop a Real-Bogus classification framework that can be trained without human-labeled data using injected transients and bogus-dominated survey data, remains robust under strong class contamination, and provides calibrated uncertainty quantification. We combine simulated transient injections with a contaminated survey class and train a dual-network model using asymmetric co-teaching for classes with different label-noise levels. We evaluate performance on a benchmark subset and analyze the learned representation with latent-space visualization tools. For uncertainty quantification (UQ), we compare MC dropout and deep ensembles and propose a low-cost hybrid strategy that exploits the dual-network setting to improve calibration. We extend the evaluation to the light-curve domain to assess recovery of light-curve classes. The method achieves strong Real-Bogus performance on the labeled subset and remains stable under severe class contamination. It recovers transient light-curve classes with high fidelity, while single-source identification is limited by ambiguity in light-curve-derived labels. Our hybrid UQ approach achieves competitive calibration relative to more expensive ensemble baselines. Latent-space analyses indicate that uncertainty aligns with the decision boundary and reveal subclasses within the bogus population. Our results show that injection-driven, weakly supervised training can enable scalable and consistent Real-Bogus classification without human-labeled training data while providing calibrated uncertainties. The method is suited for transfer to forthcoming surveys by re-running the injection-based training pipeline.
Ensemble learning methods combine multiple weak base learners to create a robust decision model, effectively analyzing feature-response relationships across various domains. However, the assumption of accurate sample-label relationships in supervised learning often breaks down in real-world datasets, leading to performance degradation due to incorrect information. The framework-specific effects of label noise on ensemble learning have not been sufficiently explored from a theoretical perspective. This article investigates the problem of learning from datasets contaminated by label noise within ensemble frameworks. We utilize bias-variance-diversity (BVD) decomposition theory to examine the impact of noisy labels on three mainstream ensemble paradigms: Bagging, Boosting, and Stacking. Our theoretical analysis characterizes the mechanisms behind performance degradation and guides the development of targeted strategies: data filtering (DF) for Bagging, sample reweighting (SR) for Boosting, and interactive feature purification (IFPS) for stacking. We validate our approaches on synthetic and real-world noisy-label benchmarks, demonstrating consistent improvements over traditional ensemble methods through extensive comparison and ablation experiments. Our findings offer actionable insights for enhancing the robustness of ensemble learning in the presence of noisy labels, thereby broadening its applicability in practical scenarios.
Guanxiong He, Jie Wang, Zhiyong Li et al.· IEEE Transactions on Neural...· 1 citation
The deployment of deep neural networks in safety-critical domains demands reliable estimates of predictive confidence, yet conventional architectures lack principled uncertainty quantification. This survey provides a structured, critical review of methods for Uncertainty Quantification (UQ) in deep learning, scoped to ensemble-based and approximate Bayesian approaches and the measures used to summarize their outputs. Relative to existing UQ surveys, our contribution is depth on efficient ensemble approximations and single-pass methods, and a unified treatment that separates the method producing a predictive distribution from the measure that summarizes its uncertainty. We organize methods into five families: Bayesian neural networks, Monte Carlo Dropout, deep ensembles, efficient ensemble approximations, and last-layer or single-pass approaches. We situate adjacent work on evidential and prior networks, conformal prediction, and post-hoc calibration, together with the decision-time tasks of out-of-distribution detection and selective prediction. For each, we examine theoretical motivation, implementation, empirical performance, and limitations. We then review ensemble diversity theory and uncertainty measures and their decompositions, contrasting the entropy decomposition with pairwise divergence measures, and consolidate evaluation methodology so that our qualitative comparisons share a common basis. We close with a brief treatment of uncertainty in large language models and open research directions, including efficient epistemic measures for classification, last-layer diversity, diversity and calibration under shift, and hybrid architectures.
H. M. Gillis, Thomas Trappenberg, MC Dropout· 0 citations
Multi-label learning (MLL) is a machine learning paradigm that aims to predict a set of labels for each instance, rather than a single class. Such tasks arise in a wide range of real-world applications and pose significant challenges, including an exponentially large output space, dependence among labels and often severe label imbalance. These challenges amplify predictive uncertainty, making reliable uncertainty quantification essential. Conformal prediction (CP) is an attractive answer: it converts model outputs into prediction regions with distribution-free, finite-sample guarantees under the sole assumption of data exchangeability. Several adaptations of CP to the multi-label setting have been proposed. Yet these vary widely in scoring constructions, output types and targeted guarantees. This review consolidates the landscape of CP adaptations for MLL. It places existing approaches under a unified framework, examining the types of outputs and guarantees they provide, where label dependencies are incorporated, and how inference cost scales with the number of labels. It provides an in-depth analysis of all approaches using common notation, identifying their key characteristics along with their practical implications and assessing their strengths and limitations. Finally, it compares approaches side-by-side, highlighting trade-offs among guarantee types, precision of regions, compactness of outputs and scalability. This article is part of the theme issue 'Advancing uncertainty quantification in AI systems'.
Deep Neural Networks (DNNs) deployed in high-risk domains, such as healthcare and autonomous driving, must be not only accurate but also understandable to ensure user trust. In real-world computer vision tasks, these models often operate on complex images containing background noise and are heavily annotated. To make such models explainable, Concept-based Explainable AI (CXAI) methods need to be assessed for their applicability and problem-solving capacity. In this work, we explore CXAI use cases in multi-label classification by training two DNNs, VGG16 and ResNet50, on the 20 most annotated labels in the MS-COCO dataset (Microsoft Common Objects in Context). We apply two CXAI methods, CRP (Concept Relevance Propagation) and CRAFT (Concept Recursive Activation FacTorization), to generate concept-level explanations and investigate the overall evaluations. Our analysis reveals three key findings: (1) CXAI highlights learning weaknesses in DNNs, (2) higher concept distinctiveness reduces label and concept confusion, and (3) environmental concepts expose dataset-induced biases. Our results demonstrate the potential of CXAI to enhance the understanding of model generalizability and to diagnose bias instigated by the dataset.
Temperature scaling is the dominant post-hoc calibration method in modern deep learning. Its theoretical justification rests on an assumption that is rarely stated explicitly: that ground-truth labels are one-hot and deterministic. In practice, labels are frequently soft, crowd-sourced, or genuinely distributional, reflecting real disagreement among human annotators rather than annotation noise. We study whether temperature scaling retains its calibration properties when this assumption is violated, and whether any resulting degradation depends on model scale. Using CIFAR-10H and ChaosNLI, two publicly available datasets with human-annotated soft label distributions, we evaluate three model scales per modality under both hard one-hot and soft distributional label targets. Across all nine configurations we find a positive soft-label calibration gap: temperature scaling calibrated on hard labels consistently underperforms an oracle calibrated directly on soft labels, with Brier Score gaps ranging from 0.002 to 0.134. The gap grows monotonically with model scale in the vision domain and on the SNLI-derived split of ChaosNLI, and is substantially larger in the language domain (mean gap 0.079) than in vision (mean gap 0.003). A scale-ordering reversal on the MNLI-derived split remains after matched-domain training; we treat it as inconclusive for the scale hypothesis and attribute it primarily to near-chance accuracy on that split. As a second post-hoc baseline, multiclass isotonic regression yields the same qualitative conclusion: positive soft-label gaps in all nine configurations, and larger gaps in language than in vision. These findings suggest that calibration protocols built on majority-vote labels systematically misstate model reliability wherever label ambiguity is structural, with direct consequences for deployment in safety-critical settings.