An Intelligent Edge-Cloud Framework for Real-Time IoT Data Analytics Using Machine Learning

Abstract

The rapid proliferation of Internet of Things (IoT) devices has resulted in an unprecedented increase in the volume of data they generate. Real-time processing and analysis of IoT data are essential for enabling timely decision-making and appropriate response actions. However, conventional cloudbased architectures are often unable to meet the stringent latency requirements of such applications. This paper presents a novel Edge-Cloud framework that facilitates real-time IoT data processing by leveraging AWS IoT Greengrass Core to deploy lightweight machine learning models at the edge while transmitting selected data to the AWS cloud for long-term storage and advanced analytics. The proposed framework implements Random Forest and XGBoost algorithms for intelligent data filtering, incorporating a confidence-based transmission policy that significantly reduces cloud-bound data traffic. The framework reduced the volume of data transmitted to the cloud by up to 91.45% and decreased inference latency by 79% compared with the conventional cloud-based approach. In addition, the framework is readily applicable to a wide range of domains, including urban infrastructure and healthcare. The effectiveness of the proposed Edge-Cloud framework was validated using the Numenta Anomaly Benchmark (NAB), where it achieved an accuracy of 96.8%, outperforming many comparable approaches reported in the literature. This intelligent edge-cloud continuum enables scalable, secure, and cost-effective real-time IoT applications by bridging local processing with cloud-based advanced analytics.