Skip to content
Conference Open access

Automated Feature Discovery in Large-Scale Databases via Deep Learning

Oct 2026 · International Conference on Machine Learning and Applications · pp. 1229-1235 · 0 citations · 20 references

TL;DR

A three-phase deep relational learning framework that combines neural representation learning with a Relational Feature Miner (RFM) to automate feature discovery from structured database schemas is introduced.

Abstract

As large-scale relational databases become common in healthcare, finance, and e-commerce, there is a growing need for scalable, automated feature engineering. Traditional methods rely on manual schema knowledge and fixed join rules, which do not work well for complex, multi-table data. This paper introduces a three-phase deep relational learning framework that combines neural representation learning with a Relational Feature Miner (RFM) to automate feature discovery from structured database schemas. In the first phase, relational schemas are converted into heterogeneous graph structures, with primary and foreign key relationships represented as edges. In the second phase, an intra-table autoencoder transforms different types of column data, numerical, categorical, and temporal, into uniform dense embeddings, removing the need for manual preprocessing. In the third phase, the RFM uses a message-passing method on the graph to automatically gather cross-table relational patterns without custom SQL. Tests in e-commerce churn, financial default prediction, and clinical readmission prediction show consistent AUC-ROC improvements of 4.4-6.3% compared to Deep Feature Synthesis baselines, while maintaining nearly linear computational scaling (O(N)) across up to 50 million records. Ablation studies show that each phase adds unique value. These findings suggest that the proposed framework is an effective and interpretable way to automate knowledge discovery in large enterprise relational databases.

Read PDF

Similar papers

#machine learning Preprint Oct 2026

Revisiting Simple Baselines for Relational Deep Learning

Relational deep learning has emerged as a powerful paradigm for learning end-to-end predictive models over relational databases by representing them as temporal heterogeneous graphs. Recent advances based on graph neural networks and Transformers have achieved strong performance across diverse domains. However, the eff...

Viet T. Nguyen, Vijay Prakash Dwivedi, Gunther Gust · 0 citations
Review Open access Sep 2026

Hybrid clustering framework for large-scale scientific literature structuring

This study addresses the challenge of organizing and interpreting large collections of scientific literature, focusing on a rapidly growing and heterogeneous AI research domain derived from object-detection-with-small-data literature. First, based on the set of keywords using Clarivate Analytics queries, over 18,500 re...

Dmytro Teplov, Artur Radzivil, Andrej Bugajev · 0 citations
#machine learning Preprint Oct 2026

RelICL: Training-free Relational Learning with Tabular Foundation Models

Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables'columns...

Simon Forbat, Rainer Gemulla · 0 citations
Book Open access Nov 2026

Files Search Relevance using Wide and Deep Networks

Unlike web-search, enterprise search queries are more contextual, documents are private and heterogeneous, and ground-truth relevance labels are scarce. Production systems at this scale have long relied on gradient-boosted decision tree (GBDT) models such as LightGBM with hand-engineered features, given their speed, in...

Rajeev Gupta, Sandipan Mandal, Kajal Gupta et al. · 0 citations
#artificial intelligence Preprint Oct 2026

RT-J: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions

This work introduces RT-J, an 86M-parameter relational foundation model (RFM) that makes accurate predictions even with hundreds of in-context examples or fewer, and highlights the importance of real-world semantics, context engineering and large-scale pretraining for RFMs.

Rishabh Ranjan, Vignesh Kothapalli, Harsh Agarwal et al. · 1 citation · ⚡1

Temporal Heterogeneous Graph Pretraining for Relational Deep Learning

This framework combines Multi-scale Time Encoding, which captures record age using learnable time scales and type-specific projections, with Rotary Time Encoding, which represents signed inter-record intervals through rotary transformations during graph propagation.

Yi-Xin Peng, Er Jin, Diego Collarana et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.