2026· E3S Web of Conferences· Vol 736, pp. 03001· 0 citations· 10 references
TL;DR
The findings conclude that transitioning to an automated framework is imperative for enhancing system scalability and reliability in agile development environments.
Abstract
Software testing is a critical component of the System Development Life Cycle (SDLC) to ensure product quality and functionality. This study conducts a comparative performance analysis between manual testing and automated testing using Katalon Studio, specifically applied to the BISA AI platform—a Massive Open Online Course (MOOC) for artificial intelligence. Following the Software Testing Life Cycle (STLC) methodology, the evaluation is based on the ISO/IEC 25010 quality framework, focusing on four key characteristics: functional suitability, performance efficiency, reliability, and usability/maintainability. A total of 72 test cases across 14 core features were executed through three iterations to ensure data consistency. The results indicate that while manual testing achieved a higher functional suitability rate of 79.16% compared to 75.00% for automation, the automated approach demonstrated significant superiority in other metrics. Automated testing recorded a performance efficiency of 85.20% (approximately 2.8 times faster than manual) and a reliability score of 94.40% by eliminating human fatigue and detecting systemic vulnerabilities such as time-outs. Overall, automated testing reached an accumulative quality score of 84.87% ("Very Good"), whereas manual testing scored 60.55% ("Good"). The findings conclude that transitioning to an automated framework is imperative for enhancing system scalability and reliability in agile development environments.
The experimental results show that the Cypress-based end-to-end test suite has short and stable execution times, and resilient data-cy attributes significantly reduce maintenance overhead when UI changes occur.
Quoc-Binh Nguyen, Truc-Ly Phan Nguyen, Ngoc Hong Tran et al.· 0 citations
An integrated reference architecture is proposed that combines TDD's fine-grained unit-level feedback loop with BDD's stakeholder-readable acceptance criteria inside a single continuous integration and continuous delivery (CI/CD) pipeline.
Urvish Gajjar· International Journal of Sci...· 0 citations
A system for evaluating the performance of participants in proficiency testing according to ISO 17043:2023 and ISO 13528:2022 was developed using the R programming language. The processing of data obtained from participants in proficiency tests is typically a time and resource intensive task; so automation and digital transformation of this activity were implemented. The developed system was entirely programmed in the R programming language using the R-Studio IDE and was validated using data from ISO 13528:2022. It starts with the reported data from the participants, which are automatically compiled from reports in Excel format. Subsequently, homogeneity and stability assessment of the proficiency test items are performed based on measurements carried out in our laboratories. The distribution of the reported data from the participants is evaluated, and outliers in the population are assessed as well. Next, the assigned value and standard deviation for proficiency evaluation are calculated in several ways (e.g., mean and standard deviation, median and MADe, median and nIQR, Algorithm A, M-estimator, Horwitz equation) to choose the most suitable one based in the distribution of the values, and the number of independent values reported. Finally, the performance of each participant is automatically evaluated, taking into account the assigned value, the standard deviation for proficiency evaluation, and the uncertainty of assigned value, using performance statistics like z-score, ζ-score, normalized error, or others from ISO 13528, as requested by the PT coordinator. The developed system has been evaluated and approved by the Quality System Task Force (QSTF) of the Inter-American Metrology System (SIM), demonstrating reliability and efficiency. This approval enables our laboratory to organize more proficiency tests annually, meeting national and international demand while ensuring quality and reliability of our Proficiency Test services.
Julia Morales, J. Gonzales, Yamil Acho et al.· 150th anniversary of the Met...· 0 citations
Test automation is vital for software release in modern times, it often results in false positives due to failed tests, which leads to the waste of precious engineer's time and, hence, loss of trust in the system. The proposed research project seeks to design a predictive model that can estimate the possibility of false positive failure before test automation. Our model was trained using the data on 12,400 test cases from 840 automated tests that were conducted during 18 months of continuous integration process. In total, there were 42 features selected as input data for training – they include data from logs, information about code changes, environmental factors, and failure history. Five machine learning algorithms were tested, XGBoost algorithm being the most successful with accuracy being 91.2%, while F1-score equalled 90.5%. Testage, code churn, and failure rate were the most important features. After deploying our tool into production CI pipeline, we noticed a reduction inthe number of test cycles of up to 34% as well as increase of defect detection speed by 60%.
Gitanjali Kumari, M. K. Shukla, Nawnit Kumar et al.· 2026 6th International Confe...· 0 citations
Algorithm-Driven Development is introduced, a methodology developed from industrial practice to address recurring challenges in translating requirements into reliable, testable, and maintainable software behavior that provides systematic coverage of functional scenarios from the outset of development.
Philippe Jawish, Pierre Evrard, Alexandre Lemerle et al.· Journal of Systems and Softw...· 0 citations
The monograph explores the key contradiction of modern software engineering: the conflict between the speed of delivery of functionality and the need to ensure the quality of complex systems. Based on the analysis of the evolution of QA practices, the systemic gaps of existing approaches to optimizing regression testing, such as impact analysis of changes (TIA) and methods based on machine learning (ML), are demonstrated. The author's methodology, Adaptive Quality and Test Impact (AQTI) Framework, is proposed as a solution. The scientific novelty of the framework is the integration of signals of the impact of changes in the code with a multifactorial assessment of the quality of the tests themselves, including their stability and business criticality. AQTI is formalized in the form of a transparent scoring model and an algorithm that allows you to create a minimally sufficient set of tests within the specified time constraints (SLA) with a guaranteed safety net of critical checks. The results of the retrospective modeling confirm that the framework is able to significantly reduce the time of test cycles while maintaining a high level of defect detection.
The monograph is addressed to researchers, engineers and architects. Due to its reliance on universal engineering artifacts, the proposed approach has not only theoretical but also practical value, being applicable in industries such as fintech, e-commerce and government systems.