A Comparative Analysis of Spark GraphX and GraphFrames for Healthcare Data Analytics

Abstract

Healthcare data analytics is essential for identifying disease patterns, understanding patient relationships, and supporting data-driven decision-making in modern healthcare systems. As healthcare datasets continue to grow in size and complexity, scalable graph-processing frameworks have become increasingly important for analyzing interconnected patient data. This paper compares two Apache Spark graph-processing frameworks, GraphX and GraphFrames, using the Centers for Disease Control and Prevention Diabetes Health Indicators dataset. A patient similarity graph was constructed by representing 70,692 patients as vertices and connecting highly similar patients through cosine–similarity relationships, resulting in 16,009,101 graph edges. The two frameworks were evaluated using the same graph structure and execution environment with respect to graph construction time, execution performance, memory consumption, application programming interface usability, and scalability. In addition to the framework comparison, graph-based features, including in-degree, out-degree, total degree, and PageRank, were extracted to examine the structure of the patient similarity network. The experimental results showed that GraphX completed graph construction, PageRank computation, and degree calculations considerably faster than GraphFrames. However, GraphFrames offered a higher-level programming interface, simpler integration with Spark SQL, and easier implementation of graph analytics workflows. The extracted graph measures also highlighted highly connected and structurally important patients within the network, demonstrating the usefulness of graph-based feature extraction for healthcare data analysis. GraphX completed graph construction approximately 33 times faster (92.5 s vs. 3037 s), PageRank computation approximately 780 times faster (9.1 s vs. 7132 s), and degree computation over 10,000 times faster (1.0 s vs. 10202 s) than GraphFrames, whereas GraphFrames used substantially less memory; these results indicate that GraphX is more suitable for performance-oriented graph workloads, whereas GraphFrames is advantageous when development flexibility and DataFrame integration are primary considerations.