Democratizing Machine Learning: A KNN-Guided Adaptive AutoML Framework

Abstract

This research presents an Automated Machine Learning (AutoML) data pipeline designed to enhance machine learning workflows for diverse data types, including structured tabular (CSV) and time-series datasets. The proposed system automates key processes such as data preprocessing, model selection, and architecture optimization. To achieve efficient model evaluation, LazyPredict is integrated for rapid benchmarking, while Artificial Neural Networks (ANNs) handle complex structured data tasks. For time-series modeling, a Long Short-Term Memory (LSTM) network is employed to capture temporal dependencies. A lightweight KNN-guided architecture recommendation mechanism is introduced to dynamically determine optimal ANN and LSTM architectures based on dataset properties. This method provides a computationally efficient alternative to conventional Neural Architecture Search (NAS) approaches, reducing model selection overhead while maintaining performance. Extensive experiments demonstrate the pipeline's effectiveness, achieving an average accuracy of 74.71% across 8 classification datasets and an average loss reduction of 91.52% across 12 time-series datasets. By automating fundamental machine learning processes, this system lowers the barrier to entry for students and researchers, enabling more accessible and efficient AutoML.