Data processing clusters dealing with big data are facing extended completion times for jobs because the RDMA feature is not being used efficiently. Our production data from a large cluster, which has many server nodes and is responsible for processing extensive data jobs, reveals that the current use of the RDMA technology is causing some jobs to finish much later than usual, with a few taking more than double the standard time to be completed. In this paper, we introduce the design and implementation of Turbo, a high-performance, scalable communication framework tailored for large-scale data processing clusters. The essence of Turbo’s strategy lies in the use of a dynamic block-level flowlet transmission system and a non-blocking communication middleware, which are designed to boost network throughput and system scalability. Moreover, Turbo maintains high system reliability by incorporating an external shuffle service with TCP as a fail-safe option and maintaining metadata management information using the NICs. We have integrated Turbo into Apache Spark and conducted evaluations on both a small-scale test environment and a large-scale cluster with hundreds of server nodes. The findings from the small-scale testbed demonstrate that Turbo enhances network throughput by 15.1% and upholds high system reliability. Additionally, the large-scale production data indicates that Turbo is capable of decreasing job completion times by 23.9% and increasing the job completion rate by $2.03\times $ compared to the current RDMA solutions. In addition, during large-scale tests, we also found that Turbo has improved the computing efficiency of the cluster and saved approximately 24.3% of the CPU utilization.
Xuya Jia, Zhiyi Yao, Chao Peng et al.· IEEE Transactions on Network...· 0 citations
FPGA-based SmartNICs are widely deployed for cloud network function acceleration, but their memory subsystem is under increasing pressure because of large session tables. Conventional wisdom suggests that high packet processing performance relies on advanced memories (e.g., SRAM, HBM), but those are costly to add at cloud scale. Dorado is a novel design that scales SmartNIC session tables entirely on inexpensive DDR modules. At the heart of Dorado are three new techniques that extract commodity DDR performance by restructuring session table layout, decomposing processing pipelines to reduce locking, and scheduling memory accesses to minimize stalls. Our testbed results show that Dorado improves packet processing rates by 33%, even with fewer hardware resources. Further, we have deployed Dorado to millions of servers, processing network traffic from billions of users on a large public cloud for over three years. Our production results show that Dorado can accommodate up to 16M session entries, reduce memory cost by 80%, while enabling 50Mpps line-rate processing.
Heng Yu, Kai Ren, Jiajun Liang et al.· Proceedings of the ACM SIGCO...· 0 citations
Modern cloud gateways have evolved to include diverse network functions and heterogeneous hardware, such as programmable switches and FPGAs, to handle increasing workloads and minimize forwarding latency. Existing network tracing tools, however, operate primarily at device granularity and cannot pinpoint which function on which hardware component causes packet losses or latency spikes. To bridge this gap, we present CubeTrace, a unified, function-level flow tracing system that enables microscopic tracing inside heterogeneous cloud gateways. CubeTrace standardizes tracing units as cubes across different hardware platforms, regardless of their varied underlying implementations, and operates at flow granularity for reliability reasons. This introduces a new tracing abstraction for heterogeneous gateways while maintaining low overhead. Moreover, the collected flow-cube data by CubeTrace can be decoded into packet-level representations and integrated with well-established distributed tracing frameworks, enabling the use of off-the-shelf analysis tools. Our evaluations demonstrate that CubeTrace introduces minimal overhead, consuming less than 1% of memory resources and adding less than 1% to forwarding latency. Having been deployed in a large-scale cloud gateway, CubeTrace has significantly improved problem localization, reducing resolution times from hours or even days to just minutes.
Yunming Xiao, Yinchao Yang, Jiaqi Zheng et al.· Proceedings of the ACM SIGCO...· 1 citation
This work proposes Turbo, a first-of-its-kind in-network aggregation system that accelerates long-context inference by offloading query broadcast and attention aggregation to switches and introduces a rolling forward scheme that propagates states to enable cross-stage updates.
Ying Wan, Yuchen Xu, Chuwen Zhang et al.· Proceedings of the ACM SIGCO...· 0 citations
XFir is the first hardware-accelerated new-flow setup system for cloud host servers that delivers high CPS throughput while preserving sufficient flexibility, and is cost-effective to deploy, requiring only a single DPU per host.
Shihan Lin, Shunqiao Jiang, Liang Wang et al.· Proceedings of the ACM SIGCO...· 1 citation