Jul 2026· International Journal for Research in Applied Science and Engineering Technology· 0 citations
TL;DR
Experimental evaluation on the Madelon dataset indicates that the automated and interpretable pipeline performs comparably to, and in some respects favourably against, baseline feature engineering approaches, demonstrating the practical effectiveness of combining scalable feature generation with interpretable AutoML.
Abstract
Feature engineering plays a key role in determining the performance of machine learning models, but manual feature
design is time-consuming and requires strong domain knowledge. This work presents an automated feature engineering
framework integrated with an interpretable AutoML pipeline, built around the BigFeat methodology. The system automatically
generates new features from existing data using mathematical and logical operators and selects the most stable and relevant
features for learning, while preserving interpretability by maintaining traceable mappings between original and engineered
features. The framework is designed to handle large, high-dimensional datasets with manageable computational overhead.
Automated model selection and hyperparameter tuning across Random Forest, Logistic Regression, and Decision Tree
classifiers are incorporated to optimize predictive performance without manual intervention. The proposed system is intended to
reduce human effort and development time in the feature engineering process while remaining scalable and adaptable to
different datasets. Experimental evaluation on the Madelon dataset, a high-dimensional synthetic benchmark for feature
selection, indicates that the automated and interpretable pipeline performs comparably to, and in some respects favourably
against, baseline feature engineering approaches, demonstrating the practical effectiveness of combining scalable feature
generation with interpretable AutoML.
AFE is an automated feature engineering system that has become a key enabler to scalable and high-performing machine learning systems with scalable systems that run on tabular data. The conventional feature engineering makes excessive use of domain, trial and error and iterative optimization, which are computed consuming time, error prone and hard to repeat. As data-driven applications in finance, healthcare, manufacturing, and e-commerce have been exponentially increasing, there is an increasing need in automated, systematic, and reliable methods of features construction. The overall objective of automated Feature Engineering methods is to generate, transform, select, and optimize features adventurously, from raw tabular data, with minimal human intervention, and at a higher predictive efficiency. The paper contains a complete detailed analysis of automated feature engineering approaches to tabular data with references to their theoretical principles, algorithmic approaches, and real-world examples. The paper expounds on rule construction based feature construction, statistical construction, deep learning based representation, evolutionary learning, and reinforcing learning methods, and end to end AutoML. An intricate literature review shows major achievements, comparisons, and unresolved issues. The suggested methodology defines the three branches of feature generation, selection and evaluation as a single automated pipeline via mathematical representation and algorithmic processes. The effectiveness of automated feature engineering is proved by experimental results that show that the method can enhance the accuracy and robustness of models as well as improve generalization with respect to the multiple benchmark datasets. Lastly, issues of limitations, interpretability, computational trade-offs, and research directions are discussed in the paper. The given publication meets the IEEE publication standards and offers well-organized, high-quality information to a researcher or an organization practitioner dealing with tabular data analytics.
Yuki Tanaka, Kenji Sato· International Journal of Art...· 0 citations
In domains such as finance, healthcare, and industry, feature engineering remains the key bottleneck limiting the performance of machine learning models on tabular data. While Automated Feature Engineering (AutoFE) aims to reduce this manual effort, existing approaches still suffer from distinct limitations: data-driven exploration can waste substantial computation on semantically meaningless feature combinations, whereas knowledge-driven approaches using Large Language Models (LLMs) struggle to construct high-order interactions without rich structural context and are typically guided only by coarse global metrics. We propose CoFE (Collaborative Feature Engineering), a two-phase framework that tightly couples search-based exploration with LLM-driven reasoning. In the exploration phase, CoFE leverages an LLM-constructed semantic feature schema to guide a Monte Carlo Tree Search (MCTS), enforcing semantic constraints while encouraging a diverse pool of complex candidate features that provides the missing structural context for LLMs. In the refinement phase, CoFE introduces feature health reports, a diagnostic artifact that supplies the LLM with actionable sample-level and structural feedback for targeted corrections. Experiments on 16 public tabular benchmarks show that CoFE consistently outperforms state-of-the-art data-driven and LLM-based AutoFE methods on the majority of datasets, while offering favorable computational efficiency.
Weihao Jiang, Ziang Nan, Zhihui Shi et al.· Proceedings of the 32nd ACM...· 0 citations
The increasing complexity of contemporary software systems makes accurate and interpretable software fault prediction a challenging task. Although ensemble learning techniques have demonstrated promising predictive performance, existing approaches continue to face challenges related to feature selection, cross-project consistency, and model interpretability. This study proposes a Hybrid Boosted Stacking Ensemble Framework (HBSEF) for software fault prediction that integrates hybrid feature selection, ensemble learning, statistical validation, and SHAP-based interpretability. The proposed framework incorporates data preprocessing, class-imbalance handling using Random Oversampling, and a hybrid feature selection strategy integrating filter, embedded, and wrapper methods to identify software metrics with consistent predictive importance across multiple software projects. The core predictive model employs CatBoost, XGBoost, and LightGBM as base learners, with XGBoost serving as the meta-learner in the stacking ensemble. The framework is evaluated on an unseen test set using standard classification metrics, while paired t-testing is employed to assess the statistical significance of performance differences. Experimental results identify LOC, CBO, WMC, RFC, LCOM, and MAX_CC as the most influential software metrics. The proposed framework demonstrates statistically significant improvements over the baseline ensemble models, as validated through paired t-testing (t = 4.53, p = 0.0004), indicating the effectiveness of the proposed approach in enhancing software fault prediction performance. SHAP-based explanations provide both global and local interpretations, revealing the contribution of individual software metrics and base-model predictions to fault classification. Overall, the findings indicate that integrating principled feature selection with a boosting-based stacking ensemble can improve software fault prediction performance while providing greater transparency for software quality management.
Harsimran Kaur, Hardeep Singh, Amitpal Singh Sohal et al.· International journal of com...· 0 citations
Structured data drives enterprise decision-making, yet building predictive pipelines for time-series and tabular modalities requires intensive feature engineering, model selection, and other ''tricks of the trade''. AutoGluon, an open source AutoML system, automates this through multi-layer stack ensembling, providing a unified API that reliably achieves state-of-the-art accuracy across time series and tabular data. Concurrently, Foundation Models for Structured Data (FM4SD) have emerged to push the boundaries of predictive performance, enabling both powerful zero-shot inference and efficient fine-tuning. Specifically, Chronos-2 delivers state-of-the-art multivariate forecasting for time series, while a rapidly growing ecosystem of tabular foundation models (e.g., TabPFN, Mitra, TabICL) transforms how practitioners execute classification and regression tasks. This system-focused tutorial teaches attendees how to operationalize FM4SDs using AutoML across temporal and tabular modalities, with an additional highlight on the newly released Chronos-2. We guide participants through a technical progression: establishing AutoGluon as the orchestration framework, detailing the internal mechanics of Chronos-2 and various tabular FMs, and unifying them into production-ready pipelines.
Boran Han, Nick Erickson, Abdul Fatir Ansari et al.· Proceedings of the 32nd ACM...· 0 citations
Over the past decade, many test adequacy metrics have been proposed for deep learning that characterize test dataset adequacy from different perspectives, e.g., neuron activation behavior, latent feature coverage, decision-boundary exploration, etc. However, these metrics are typically released as independent research prototypes with substantially different installation and preprocessing requirements, execution workflows, and configuration mechanisms. These complications make them quite difficult to reproduce, compare, and adopt in research work and practical deployment alike. In this paper, we present the engineering details of ADEPT, a framework that integrates representative adequacy techniques, including neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow. ADEPT provides a template-based metric interface with well-defined extension points for integrating new adequacy metrics. Furthermore, it provides YAML-based configuration management, preprocessing-cache reuse, and structured result reporting, making it easy to use in any research and development workflows. ADEPT is designed for researchers and practitioners who wish to reproduce and apply adequacy metrics without spending days or weeks implementing missing tooling or configuring disparate research prototypes. A demo video is available at https://aub.ie/ADEPT_video.
Yidi Kao, Shawn Burnham, Tommi Rose Fahy et al.· 0 citations
Evaluating the performance of machine learning workflows is a major computational bottleneck in automated machine learning (AutoML), particularly for complex pipelines involving preprocessing, model selection, and hyperparameter optimization. This work aims to develop an efficient performance prediction framework that estimates the expected accuracy of candidate machine learning workflows on unseen datasets without requiring explicit model training. We formulate performance prediction as a meta-learning regression problem that leverages historical experimental results from the OpenML platform. Machine learning workflows are represented as structured pipelines and encoded using text-based vectorization techniques, including TF-IDF, count-based, and hashing vectorizers, as well as LLM BERT embeddings. These workflow descriptors are combined with dataset-level meta-features capturing basic statistical and structural properties. Several regression models are evaluated as meta-learners, including linear models, decision trees, random forests, Gaussian processes, and gradient-boosted decision trees. The approach is systematically evaluated on the OpenML-CC18 benchmark suite using cross-validation over more than 100,000 workflow-dataset evaluations. The proposed framework achieves strong predictive performance across a wide range of workflows and datasets. In particular, gradient-boosted decision tree regressors combined with standard TF-IDF representations of workflows consistently yield the best results, reaching an average coefficient of determination \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document} of approximately 0.8 on unseen test data. While transformer-based MiniLM embeddings were evaluated, they did not consistently outperform sparse TF-IDF representations and incurred higher computational cost. Feature ablation studies indicate that restricting vocabulary size degrades performance, while extending representations with bigrams provides only marginal gains at substantially higher computational cost. The results demonstrate robust generalization across heterogeneous workflows and dataset characteristics. This study shows that accurate performance prediction for classical machine learning workflows can be achieved through meta-learning using readily available OpenML meta-data. The proposed approach enables rapid and computationally efficient estimation of workflow performance, making it well suited for accelerating AutoML search and model selection. The results indicate that careful selection of regression models is more critical than increased representational complexity, with simple and scalable workflow encodings yielding the most robust performance. Given its scalability and flexibility, the framework provides a strong foundation for future extensions incorporating richer dataset descriptors, larger meta-datasets, and more expressive embedding and regression models.
Roman Neruda, Juan Carlos Figueroa–García, Carlos Franco· SN Computer Science· 0 citations