Jul 2026· International Conference on Artificial Intelligence Testing· pp. 193-196· 0 citations· 11 references
Abstract
Test Case Prioritization (TCP) aims to reorder test cases to enable earlier fault detection in Continuous Integration pipelines. Machine learning-based TCP approaches have shown promising results but require substantial historical execution data, creating a cold-start challenge for new or rapidly evolving projects. This paper presents an empirical study of direct transfer learning, where models trained on data-rich source projects are applied to unseen targets without any retraining. A total of 578 experiments are conducted across 17 Defects4J projects (34 within-project baselines and 544 cross-project transfer scenarios) using 270,611 feature vectors. Results show that with appropriate source selection, cross-project transfer can match or surpass within-project baselines. A three-way feature ablation (HIS-only, COV-only, full 13-feature) finds that no feature subset consistently exceeds random ordering (E[APFD]≈0.5) on average, confirming that source selection is the critical variable. Analysis identifies failure-rate alignment and source dataset richness as key predictors of transfer quality, providing source-selection criteria that require no target-side data collection.
In cross-project defect prediction (CPDP), divergence in feature distribution between the source and target often fails when predicting defects across unrelated projects. Such a mismatch is typical rather than exceptional in real deployment scenarios. To address this, the Transfer Component Analysis (TCA) technique projects the source and target into a shared subspace, where Maximum Mean Discrepancy is minimized. However, combining fixed ensemble classifiers with TCA on NASA datasets is yet to be explored. Previous studies either searched ensemble compositions adaptively or conflated alignment with source selection. In this study, we trained a two-layer hybrid ensemble of Bagging and AdaBoost classifiers, alongside a Logistic Regression meta-learner, on TCA-aligned features across 20 directed source–target pairs. We used five PROMISE datasets, each having 21 McCabe and Halstead features. Experiments were conducted, and the findings show that, against an unaligned baseline using the same ensemble, TCA alignment increased mean AUC by 0.131 (0.625 to 0.755), mean F1 by 0.155, and mean MCC by 0.125. Wilcoxon signed-rank tests confirmed significance across all three metrics (p < 0.003, rank-biserial r = 0.714, Cliff’s Delta d ≥ 0.545). A full ablation showed that alignment was the dominant contributor to performance, with TCA improving AUC by +0.131 over the unaligned baseline. In contrast, SMOTE traded a small AUC reduction (−0.014) for substantial F1 gains (+0.134), while the stacking layer provided modest improvements in F1 and MCC. TCA reduced MMD across all 20 source–target pairs by a mean of 83.3%. Sensitivity analysis further showed that the TCA subspace dimensionality parameter, k, exhibited non-monotone, pair-specific AUC sensitivity, with optimal values ranging from 5 to 30. These findings indicate that alignment parameter selection substantially influences CPDP performance and that a fixed global default is inadequate.
Direct-copy and negative-control analyses show that gains depend on source-target compatibility and LLM adaptation; stored source accuracy alone does not predict transferability.
Kabir Baghel, R. Timofte, Dmitry Ignatov· 0 citations
Continuous integration (CI) environments are increasingly challenged by the rapid growth in test cases and code changes, rendering exhaustive regression testing impractical. In our previous work [1], [2], we introduced TCP-Net and TCP-Net++, deep neural network models for test case prioritization that showed promising improvements in fault detection. However, these models struggled to fully capture the sequential dependencies inherent in evolving CI systems. In this paper, we present TE-TCP-Net, a novel transformer-based model that leverages parallel transformer encoders to dynamically learn from historical test execution data, source code modifications, and test case characteristics. By effectively modeling sequential relationships across multiple CI builds, TE-TCP-Net achieves superior performance in terms of Average Percentage of Faults Detected (APFD) and Normalized APFD (NAPFD) compared to state-of-the-art methods. The results show that TE-TCP-Net is very effective, indicating its potential for practical usage in real-world scenarios. We validate our approach on two industrial datasets, Calibre PERC and Calibre xRC, where TE-TCP-Net significantly improves early fault detection and overall testing efficiency, establishing a new benchmark for test case prioritization in dynamic CI environments.
Organizations maintain task-specific adapters for open-weight language models, and each new base-model release forces a migration decision: retain existing specialists, port adapters, refresh from preserved behavior, or retrain. Prior transfer work evaluates isolated model pairs, without studying these choices across real model release sequences. We present UpgradeBench, a decision-driven longitudinal benchmark covering four consecutive Qwen releases, one continuation checkpoint, six tasks, and two model sizes, augmented by OLMo checkpoints with known training lineage. The benchmark disentangles three core questions: whether a new checkpoint improves fixed-recipe retrained specialist performance, whether specialization assets transfer across versions, and what recovery resources are usable. We observe upgrade gains differ across task-scale-release episodes: some retrained baselines improve while others stay within training noise, with durability ranging from under one release interval for text-to-SQL to over fourteen months for intent classification. Direct adapter copying depends neither on architecture nor model family: on OLMo, retention drops from 0.88-0.99 at 46B-token continued pretraining to zero at 2.9T tokens; annealing and model souping introduce no extra harm, with portability decaying with continued-pretraining distance. Given preserved input data, teacher relabeling recovers target-base specialists without fresh gold annotations, though compute savings are not guaranteed. Simulating a fixed decision policy over 33 upgrade episodes yields 0.37pp mean quality regret with zero behavioral regressions at one-third the compute and label cost of full retraining. A lightweight CKA probe over 256 prompts predicts cross-version adapter portability (Spearman 0.74 across eight model pairs). We release per-example predictions, cost logs, split manifests, and evaluation code.
: Automatic bug triage is a critical software maintenance task that reduces the high human resource cost of manual developer assignment. In this paper we evaluate various machine learning approaches on Eclipse dataset. We compare classical machine learning approaches like TF IDF + XGBoost, LogReg, NaiveBayes and modern transformer based architectures including: Bert, Nomic, Mpnet, Jina, BGE, E5, CodeLlama, Llama-3.1-8B-Instruct, ChatGPT few-shot prompting. We compare various approaches including using transformer based architectures with frozen weights as embedding with a fine-tuned neural network classification head (MLP). For maximal performance we selected the best performing model E5 (47.63% accuracy) and fully fine-tuned it and achieved 72.49% top-1 accuracy which is the best performing deep learning based approach to the best of our knowledge slightly under performing our best approach: TF IDF+XGBoost 76.47%. Furthermore we also experienced with adding the embedding vectors of some of the aforementioned transformer-based models to the best performing TF IDF+XGBoost models but we experienced slightly worse performance than with only TF IDF+XGBoost. We also experimented with Retrieval Augmented Generation (RAG) and achieved 36.81% top-1 accuracy. The RAG results were primarily influenced by the retrieval rather than the generator.
Márk Lajkó, Balázs Nagy, László Vidács· Proceedings of the 21st Inte...· 0 citations
Test case prioritization is the process of ordering test cases based on their importance to improve regression testing efficiency. Test case prioritization approaches have been demonstrated to improve regression testing processes. But operating the whole regression test suite can be inconvenient and costly, particularly for large systems. To overcome this problem, a novel An Intelligent Deep Learning Model for Automated Test Case Prioritization Systems using Neural Networks (Deep-TCP) has been proposed. Test cases are collected from the source code repository and split into test steps. The test steps are preprocessed using Normalization, Tokenization and stemming to remove noise. After pre-processing, word embedding is computed and the embedded features are clustered using K-means clustering. The novelty of the proposed integration is where the effort is of word embedding, K-means clustering, Radial Basis Function Network (RBFN), and CNN-BiGRU to capture both semantic and sequential connections between test cases. Radial Basis Function Network (RBFN) is used for extracting the relevant features and Convolutional Neural Network-Bidirectional Gated Recurrent Unit (CNN-BiGRU) is used for testcase priority such as high priority, average priority and low priority. The proposed framework has been implemented using Python (PyTorch) and evaluated on a system with NVIDIA RTX 4090 GPU. The efficacy of the proposed Deep-TCP framework has been determined using evaluation metrics such as Average Percentage of Faults Detected (APFD). Average APFD of the proposed method is 94.5% which is higher than 71.25% at ATRL-TCP, 73.5% at QAOA-TCS and 81.5% at BootQA approaches.
S. S, Shashi Mehrotra· international journal of eng...· 0 citations
