Jul 2026· Anais do XXXIX Concurso de Teses e Dissertações da SBC (CTD-SBC 2026)· 0 citations· 30 references
TL;DR
This thesis introduces the Dynamic Pipeline CASH problem, which extends the CASH formulation to incorporate meta-model-driven search space creation for pipelines, using Metalearning (MtL) to dynamically build task-specific search spaces.
Abstract
Although Automated Machine Learning (AutoML) systems allow the use of Machine Learning (ML) to automate the design of ML pipelines, they typically search over fixed, task-agnostic configuration spaces, leading to high computational costs. This paper overviews a Ph.D. thesis that proposes a paradigm shift: using Metalearning (MtL) to dynamically build task-specific search spaces. Unlike prior approaches that either optimize within a fixed search space or directly recommend algorithms without an optimization step, this thesis introduces the Dynamic Pipeline CASH problem, which extends the CASH formulation to incorporate meta-model-driven search space creation for pipelines. The thesis contributes a systematic literature review identifying meta-knowledge as the unifying thread across AutoML subfields, applied studies reinforcing the importance of algorithm selection and tuning, a large-scale benchmark of over one million pipeline configurations, and the pymfe package for reproducible meta-feature extraction. These building blocks converge into a novel MtL framework that dynamically reduces search spaces while maintaining competitive performance.
This work presents LACE, an AutoML framework that instead searches over complete executable pipeline programs: an evolutionary loop maintains a population of scikit-learn-compatible Python classes, and a large language model acts as the variation operator.
Sofoklis Kitharidis, C. Veenman, J. V. Rijn et al.· 0 citations
Structured data drives enterprise decision-making, yet building predictive pipelines for time-series and tabular modalities requires intensive feature engineering, model selection, and other ''tricks of the trade''. AutoGluon, an open source AutoML system, automates this through multi-layer stack ensembling, providing a unified API that reliably achieves state-of-the-art accuracy across time series and tabular data. Concurrently, Foundation Models for Structured Data (FM4SD) have emerged to push the boundaries of predictive performance, enabling both powerful zero-shot inference and efficient fine-tuning. Specifically, Chronos-2 delivers state-of-the-art multivariate forecasting for time series, while a rapidly growing ecosystem of tabular foundation models (e.g., TabPFN, Mitra, TabICL) transforms how practitioners execute classification and regression tasks. This system-focused tutorial teaches attendees how to operationalize FM4SDs using AutoML across temporal and tabular modalities, with an additional highlight on the newly released Chronos-2. We guide participants through a technical progression: establishing AutoGluon as the orchestration framework, detailing the internal mechanics of Chronos-2 and various tabular FMs, and unifying them into production-ready pipelines.
Boran Han, Nick Erickson, Abdul Fatir Ansari et al.· Proceedings of the 32nd ACM...· 0 citations
Evaluating the performance of machine learning workflows is a major computational bottleneck in automated machine learning (AutoML), particularly for complex pipelines involving preprocessing, model selection, and hyperparameter optimization. This work aims to develop an efficient performance prediction framework that estimates the expected accuracy of candidate machine learning workflows on unseen datasets without requiring explicit model training. We formulate performance prediction as a meta-learning regression problem that leverages historical experimental results from the OpenML platform. Machine learning workflows are represented as structured pipelines and encoded using text-based vectorization techniques, including TF-IDF, count-based, and hashing vectorizers, as well as LLM BERT embeddings. These workflow descriptors are combined with dataset-level meta-features capturing basic statistical and structural properties. Several regression models are evaluated as meta-learners, including linear models, decision trees, random forests, Gaussian processes, and gradient-boosted decision trees. The approach is systematically evaluated on the OpenML-CC18 benchmark suite using cross-validation over more than 100,000 workflow-dataset evaluations. The proposed framework achieves strong predictive performance across a wide range of workflows and datasets. In particular, gradient-boosted decision tree regressors combined with standard TF-IDF representations of workflows consistently yield the best results, reaching an average coefficient of determination \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R^2$$\end{document} of approximately 0.8 on unseen test data. While transformer-based MiniLM embeddings were evaluated, they did not consistently outperform sparse TF-IDF representations and incurred higher computational cost. Feature ablation studies indicate that restricting vocabulary size degrades performance, while extending representations with bigrams provides only marginal gains at substantially higher computational cost. The results demonstrate robust generalization across heterogeneous workflows and dataset characteristics. This study shows that accurate performance prediction for classical machine learning workflows can be achieved through meta-learning using readily available OpenML meta-data. The proposed approach enables rapid and computationally efficient estimation of workflow performance, making it well suited for accelerating AutoML search and model selection. The results indicate that careful selection of regression models is more critical than increased representational complexity, with simple and scalable workflow encodings yielding the most robust performance. Given its scalability and flexibility, the framework provides a strong foundation for future extensions incorporating richer dataset descriptors, larger meta-datasets, and more expressive embedding and regression models.
Roman Neruda, Juan Carlos Figueroa–García, Carlos Franco· SN Computer Science· 0 citations
Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance. A critical yet largely overlooked component of these pipelines is the feature function that maps problem instances to inputs for machine learning models. Existing L2O methods typically rely on hand-crafted features, making representation design manual and largely fixed across domains. We introduce FunL2O, the first unified framework for automating feature design through LLM-driven program evolution for L2O. In a FunSearch-style loop, an LLM proposes executable feature functions, while a fixed evaluation process retrains the original L2O model and measures downstream optimization performance. We evaluate FunL2O on linear and quadratic programming tasks involving solution prediction and warm-starting, as well as on mixed-integer optimization tasks using GNN-guided backdoor branching and Predict-and-Search. Across continuous and discrete optimization tasks and four LLMs, the evolved features consistently outperform hand-crafted representations. These results establish LLM-driven feature evolution as a general and effective approach to automating representation design in L2O.
Bingheng Li, Junyang Cai, Yupeng Zhang et al.· 0 citations
Optimus is a general-purpose model transformation framework built in the PyTorch 2.x (PT2) machine learning compiler that applies an efficient greedy search algorithm for pattern matching and replacement, while preserving model semantic.
Menglu Yu, Jiaqi Xu, Yuzhen Huang et al.· Proceedings of the 32nd ACM...· 0 citations
Experimental evaluation on the Madelon dataset indicates that the automated and interpretable pipeline performs comparably to, and in some respects favourably against, baseline feature engineering approaches, demonstrating the practical effectiveness of combining scalable feature generation with interpretable AutoML.
Ms. CH. Vasavi, Ms. SK. Raqeeba· International Journal for Re...· 0 citations
