Single-Core Hotspots on Your VNF? Break Them Up!

𝜇 slice: Toward Accurate Sub-Core Allocation with eBPF

DPIO: A Unified I/O Architecture for Heterogeneous CPU and DPU NVMeoF

To alleviate the overhead of moving massive data in modern data centers, Data Processing Units (DPUs) are increasingly deployed to offload NVMe-over-Fabrics (NVMeoF) remote storage access from the CPU. However, current heterogeneous CPU-DPU systems suffer from inflexible resource partitioning, leading to suboptimal utilization of computational power and network bandwidth. In this paper, we present DPIO, a unified I/O processing stack designed to harmonize the collaboration between CPU and DPU in NVMeoF environments. By implementing fine-grained task orchestration, DPIO effectively bridges the performance gap between DPU offloading and CPU processing, achieving near-optimal system performance across diverse workloads. Our real-world implementation demonstrates that DPIO delivers over 2 × higher throughput compared to conventional DPU-centric NVMeoF approaches, significantly enhancing the efficiency of data-intensive applications.

CoCoScale: Leveraging Layer-wise Scaling to Unlock the Potential of Online LLM Serving

CoScale is proposed, a layer-wise dynamic scaling mechanism that selectively expands the parallelism of hot layers onto idle resources reclaimed from underutilized devices, enabling elastic data parallelism without altering model architectures or adding hardware overhead.

Honey, I Shrunk the Headers With Flow.ZIP

Packet header overhead is a persistent source of inefficiency in packet-switched networks, reducing goodput and increasing network load. Trends like tunneling further increase this overhead, significantly impacting flow completion times. While, in principle, it is possible to compress these headers, existing methods require specialized hardware on every hop to compress/decompress the packet to/from custom header formats. In this paper, we present Flow.zip, a backward-compatible header compression mechanism designed for existing data center networks. Our solution leverages a combination of last-hop network offload and MPLS support, both of which are ubiquitous in modern data center deployments. Flow.zip overcomes scalability limitations in these components by selectively and intelligently coordinating compression for a subset of flows. Doing so, Flow.zip achieves up to 58% reduction in average flow completion time on real-world data center workloads.

CCSwitch: A Scalable Data Plane for Non-Blocking In-Network Collective Communication

Collective communication operations in AI and HPC workloads generate heavy network traffic. Offloading these operations to network switches reduces latency, but performing arithmetic and replication at line rate is difficult, especially as port counts and link speeds grow. Existing in-network approaches rely on accumulation buffers that not only limit throughput but also require complex state management to handle stragglers and congestion. We present CCSwitch, a modular switching fabric built from 4×4 non-blocking Collective Engines (CEs). Each CE combines spatial and temporal parallelism to perform reductions without accumulation buffers. CEs compose into k-ary n-tree topologies, scaling to 32- and 256-port switches while preserving non-blocking throughput. Source routing and flit-level synchronization keep per-switch state minimal. Our FPGA implementation shows that CCSwitch's quaternary-tree reduction fabric uses up to 23% fewer LUTs and 12–30% fewer flip-flops than a comparable Clos-based design at equal throughput. Enabling the full feature set—source routing, replication, and time-multiplexed VCs—uses 1.4–1.8× more LUTs than the circuit-switched baseline, well below the 3–5× overhead typical of packet-switched NoC routers, while supporting concurrent collectives on shared links.

StrataCL: Fabric-Native Communication Library for Production Supernodes

Modern distributed AI workloads run across hundreds of accelerators, making communication a major bottleneck. Existing communication libraries remain largely buffer-centric because user and communication buffers are managed separately, causing redundant data copies or costly user-buffer registration. This paper presents StrataCL, a zero-redundancy and fabric-native communication library for production supernodes. StrataCL introduces registration-on-allocation to realize user-buffer direct communication, and designs communication operators with workload-balanced NPU-core partitioning and NPU-driven SDMA offloading to exploit supernode architecture features. On the Huawei CloudMatrix384, StrataCL improves collective bus bandwidth by up to 1.6x and improves MoE dispatch/combine bus bandwidth by up to 1.4x. Across three production workloads, StrataCL improves LLM inference throughput by 1.9x, reduces P99 TTFT by 2.2x, and reduces LLM and Recsys training iteration time by 1.4x and 1.3x, respectively.