This work proposes a novel meta-learning setting that avoids model inversion by jointly leveraging pre-trained models and unlabeled data and introduces a task-weighting mechanism based on task confidence and class distribution balance to ensure effective meta-learning.
Abstract
Meta-learning without labeled data is crucial for real-world applications, where obtaining labeled datasets can be expensive or restricted due to privacy concerns. Data-Free Meta-Learning (DFML) addresses this challenge by leveraging pre-trained models without access to training data. However, existing DFML methods rely on model inversion to generate training data, a process that is generally difficult and computationally expensive due to the need to generate high-dimensional data matching the original distribution. To address this limitation, we propose a novel meta-learning setting that avoids model inversion by jointly leveraging pre-trained models and unlabeled data. Our method generates meta-training tasks by assigning soft labels from pre-trained models to unlabeled data. Since the quality of these tasks can vary, we introduce a task-weighting mechanism based on task confidence and class distribution balance to ensure effective meta-learning. Extensive experiments demonstrate that our approach substantially reduces computational cost and improves generalization, achieving up to 104-fold speedup and 8.4 percent to 36.4 percent improvements in few-shot classification accuracy compared to state-of-the-art DFML methods.
Multi-Label Learning (MLL) refers to inducing multi-label prediction models from the precisely labeled training dataset. However, in many real-world scenarios, e.g., crowdsourcing annotations, the training datasets are often only partially valid, where each training instance is associated with a candidate label set, covering ground-truth labels but also with irrelevant ones. Naturally, learning with such datasets, formally referred to as Partial Multi-label Learning (PML), involves many noisy supervised signals, hence imposing a significant challenge to the prediction model induction. To meet this challenge, we purify the noisy supervised signals by formulating the latent label distribution, i.e., the probability of a candidate label being a ground-truth one, and then jointly learn it with the prediction model by minimizing their regularized Wasserstein distance, i.e., a robust distance for distributions as well as involving label correlations. Therefore, we propose a novel PML method, namely Wasserstein Partial Multi-Label Learning with dual Label Correlation Perspectives (Wpml3cp), solved by the gradient descent with an augmented Lagrange multiplier technique. To further enhance the robustness of Wpml3cp against exceptionally high ratios of irrelevant labels, we extend it with a Dual-branch Competitive Cleansing mechanism, leading to Wpml3cp-D. Besides, we also analyze the generalization error bound and time complexity of Wpml3cp and Wpml3cp-D. The extensive experiments are constructed by comparing Wpml3cp and Wpml3cp-D with existing PML baselines across synthetic and real-world datasets, and empirical results demonstrate that Wpml3cp and Wpml3cp-D can outperform the PML baselines in various noisy levels.
Ximing Li, Yuanchao Dai, Bing Wang et al.· ACM Transactions on Knowledg...· 0 citations
C-Score, a compact framework that evaluates training behavior in three complementary spaces: prediction, feature representation, and optimization, suggests that clean accuracy alone is insufficient for evaluating SSL robustness in open-world environments, and that internal diagnostic signals are necessary for more reliable robustness assessment under unlabeled contamination.
Tsao-Lun Chen, Chicheng Fu, Han-Yi Chou et al.· 0 citations
A critical challenge in deploying online HD map construction systems to real-world scenarios is the scarcity of labeled training data, which limits model generalization in diverse environments. To address this limitation, we propose a teacher-student semi-supervised learning (SSL) framework that generates high-quality pseudo-labels from unlabeled data through confidence-aware map refinement. Our approach first trains a teacher model on limited labeled data, then leverages Beta-distribution-based confidence maps to assess the reliability of predicted map elements across temporal observations. Unlike conventional filtering methods that discard entire elements, we introduce a spatial clipping technique that selectively preserves high-confidence regions while removing unreliable segments. The refined map elements serve as map priors that improve the teacher model's prediction accuracy on unlabeled data in a second pass. These enhanced predictions become pseudo-labels for training a student model from scratch, followed by fine-tuning on the original labeled data. Experimental results on the nuScenes dataset demonstrate that our teacher-student framework with refined pseudo-labels improves performance by +6.1 mAP under a low-label regime compared to training on labeled data alone, offering a practical solution to the labeled data scarcity problem in online HD map construction.
Chikao Tsuchiya, Dhaval Bhanderi, David Ilstrup et al.· 0 citations
Multi-label complementary label learning (MLCLL) is a weakly supervised paradigm that addresses multi-label learning (MLL) tasks using complementary labels (i.e., irrelevant labels) instead of relevant labels. Existing methods typically adopt an unbiased risk estimator (URE) under the assumption that complementary labels follow a uniform distribution. However, this assumption fails in real-world scenarios due to instance-specific annotation biases, making URE-based methods ineffective under such conditions. Furthermore, existing methods un-derutilize label correlations inherent in MLL. To address these limitations, we propose ComRank , a ranking loss framework for MLCLL, which encourages complementary labels to be ranked lower than non-complementary ones, thereby modeling pairwise label relationships. Theoretically, our surrogate loss ensures Bayes consistency under both uniform and biased cases. Experiments demonstrate the effectiveness of our method in MLCLL tasks. The code is available at https://github.com/JellyJamZhu/ComRank.
Jin Zhu, Yi Gao, Miao Xu et al.· Neural Information Processin...· 0 citations
Zero-shot classification enables models to assign labels to unseen classes without task-specific training and has become increasingly effective with the advent of Large Language Models (LLMs). However, existing zero-shot classification approaches typically present all candidate labels as a flat, unstructured list within the prompt. As the number of classes grows, this formulation exacerbates well-known limitations of LLMs, including attention dilution and positional bias, ultimately degrading classification performance. We present Label Space Reduction (LSR), a test-time training method for transductive zero-shot LLM classification. LSR uses LLM-generated pseudo-labels to iteratively refine the classification label space by systematically ranking and reducing candidate classes, enabling the model to concentrate on the most relevant options. By leveraging unlabeled data with the statistical learning capabilities of data-driven models, LSR dynamically optimizes the label space representation at test time. Our experiments across seven benchmarks demonstrate that LSR improves macro-F1 scores by an average of 7.0% (up to 14.2%) with Llama-3.1-70B and 3.3% (up to 11.1%) with Claude-3.5-Sonnet compared to standard zero-shot classification baselines. To reduce the computational overhead of LSR, which requires an additional LLM call at each iteration, we propose distilling the model into a probabilistic classifier, enabling lightweight deployment without repeated LLM calls.
Nathan Vandemoortele, Bram Steenwinckel, F. Ongenae et al.· Discover Computing· 0 citations