2026· International Journal of Machine Learning and Predictive Analytics· 0 citations
TL;DR
The results confirm that transfer learning is a core solution to the real-world problems that may be limited by available labeled data.
Abstract
Transfer learning has become an effective paradigm in machine learning and deep learning, especially in the case when labeled data are limited. Small scale data are a big challenge to the conventional deep learning models since they overfit, are not able to generalize, and do not learn features well. Transfer learning helps to address these problems by using the information of large domains in the source task to facilitate activities in target domains with scarce data. In this paper, the researcher will take the task of thoroughly examining transfer learning methods with small-scale datasets in mind. Our view of the base concepts, architecture, and strategies of domain adaptation techniques, which make the reuse of pretrained models effective are analyzed. The literature review carried out in the paper examines available literature indicating supervised, unsupervised, and semi-supervised transfer learning techniques. In addition, we suggest a systematic approach to the execution of transfer learning pipelines such as feature extraction, fine-tuning schemes, regularization schemes and metrics of evaluation. The practical experience and performance evaluation shows that, transfer learning leads to a high convergence speed, accurate classification, and robustness, in comparison to scratch training. There are negative transfer, domain-shift, and model-selection challenges which are discussed too. Lastly, the paper discusses future research directions in the area of self-supervised learning, few-shot learning, and adaptive transfer mechanisms in the regimes of small data. The results confirm that transfer learning is a core solution to the real-world problems that may be limited by available labeled data.
Experimental findings show that pretrained models outperform those trained from scratch in terms of accuracy, convergence speed, and robustness, and a unified framework is proposed to integrate both processes in a deep learning pipeline.
Kwame Nkosi· International Journal of App...· 0 citations
Transfer learning is particularly useful in settings with limited training data, and within image classification it is common to transfer learn upon massive datasets like ImageNet , CIFAR-100, or COCO . Qualitatively, it seems a transfer learning dataset should have both more classes and more examples per class than the fine tuning dataset; however, a quantitative method to choose the best transfer learning dataset does not currently exist. In this paper, we design TLDChoiceNet, a model to choose the best transfer learning dataset given a fine tuning dataset by predicting the test-set accuracy after fine-tuning. A simple version 1 achieves 0.154 MSE on the test dataset, while a version 2 leveraging an ImageNet pre-trained ResNet50 v2 embedding with per-class information attains a 5X lower MSE of 0.031. We further design two metrics that enable an unsupervised method of choosing an optimal transfer learning dataset: distribution distance (DD), which linearly regresses against fine-tune accuracy with an R2 of 0.89, and average class correlation (ACC), which improves the R2 to 0.97. Our results underscore that a dataset's low-level statistics can explain the transfer learning effect, and that using a pre-trained ImageNet can embed different classes further apart in latent feature space.
The concept of learning to learn has been dubbed as meta-learning and has proven to be an effective paradigm that allows machine learning models to learn new tasks very quickly with as little data as possible. Classical deep learning methods often need large labeled datasets and need retraining to operate again in a new setting or task, which restricts their use in dynamic and data-sparse conditions or real-time methods. Meta-learning algorithms solve this issue by learning transferable knowledge over a distribution of tasks and so enabling models to generalize with high efficiency to unknown tasks during rapid learning processes. The present paper researches in-depth the meta-learning algorithm of rapid model adaptation focusing on theoretical background, algorithm frameworks, and implementation. Our literature survey presents the various optimization based, metric based and model based methods of meta-learning with their strengths and limitations. It is suggested to formulate the meta-training and meta-testing process in a unified methodology that encompasses both the task-level optimization and the parameter initialisation strategies as well as adaptation dynamics. Whereas experimental analysis at few-shot classification benchmarks has shown the success of meta-learning to provide increased convergence speed, data-efficiency, and generalization. The findings prove that meta-learning algorithms are much better performing than the traditional transfer learning methods under low-data conditions. Lastly, the main challenges, unresolved issues in research, and future directions are addressed, such as the aspect of scalability, stability, and the implementation of the system in practice.
F. Lopes· International Journal of Mac...· 0 citations
This study systematically compares seven pre-trained feature extractors across three architectural families, convolutional neural networks (CNNs), Vision Transformers (ViTs), and self-supervised models to provide practical guidance on model selection for downstream deep learning tasks.
Rafeek Sibrikhan, M. Mufassirin· Sri Lankan Journal of Techno...· 0 citations
A transfer parameterization is developed and validated for GNNs trained with SGD, Adam, and AdamW that yields stable feature updates, learning rate transfer, and improved performance as width and depth increase, and provides a practical recipe for scaling GNNs across a variety of learning tasks and training scenarios.
Experimental results demonstrate that SSL outperforms traditional supervised learning in accuracy, scalability, and transferability, while also reducing data labeling costs.
Shalini Gupta· International Journal of Inn...· 0 citations