Behavioral-Augmented Machine Learning for Relationship Discovery in Enterprise Data Fabrics

Abstract

Integrating data across modern enterprise environments remains a significant challenge, largely because relationships between heterogeneous and distributed datasets are rarely defined explicitly. Most existing approaches to schema matching and dataset discovery depend on structural and statistical similarity measures, yet they treat data as static and overlook the behavioral patterns that naturally emerge through repeated system usage. This paper introduces an Autonomous Data Fabric component that addresses these limitations through metadata-driven relationship discovery and behavioral drift monitoring. The proposed system builds a seven-dimensional feature vector that combines structural similarity, statistical value overlap, and four behavioral signals - join frequency, co-query frequency, lineage proximity, and relationship stability - to generate calibrated confidence scores for candidate dataset joins. A hybrid scoring architecture pairs Logistic Regression with Random Forest through a weighted ensemble, supported by a deterministic fallback that maintains operational continuity when trained models are unavailable. Evaluation results show 85% end-to-end detection accuracy with perfect recall, and the integration of behavioral signals raises prediction confidence from 30.8% to 89.0%, representing a relative improvement of 188.9%. These results suggest that combining behavioral context with drift-aware monitoring meaningfully strengthens the robustness, adaptability, and governance readiness of data integration at enterprise scale.