Evaluating the Performance of Serverless Data Analytics Platforms for Large-Scale Data Processing
TL;DR
The findings indicate that cloud data warehouses provide better performance for intensive analytical workloads, while SQL-on-Lake architectures remain suitable for flexible and cost-effective data management.
Abstract
— The rapid growth of data-intensive applications has increased the demand for scalable and efficient cloud analytics platforms. This study evaluates the performance of SQL-on-Lake and traditional cloud data warehouse architectures for analytical workloads. The evaluation compares selected platforms based on query execution time, throughput, response latency, scalability, cost efficiency, and performance stability. A benchmark-driven experimental approach was adopted using standardized analytical workloads, and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method was applied for overall performance ranking. The results show that Google BigQuery achieved the best overall performance, recording the lowest query execution time, highest throughput, superior scalability, and improved cost efficiency, followed by Snowflake and Amazon Athena. The findings indicate that cloud data warehouses provide better performance for intensive analytical workloads, while SQL-on-Lake architectures remain suitable for flexible and cost-effective data management. The study recommends selecting cloud analytics platforms based on workload requirements, optimizing SQL-on-Lake environments through effective data management techniques, and adopting hybrid approaches where both flexibility and performance are required.