This letter proposes cross-IP request coalescing, which relocates the fan-out point from the guest to the SPDK vhost-user backend, and achieves up to 1.78x lower latency than the per-device baseline, with the benefit growing as concurrency increases.
Abstract
Cloud data centers rely on virtualization technologies to serve AI workloads in multi-tenant environments. With the growing scale of data-intensive AI workloads, the performance of storage I/O paths at the virtualization layer has become a critical factor. A single user request often crosses multiple IP blocks, where functional units such as storage, GPU, and accelerator devices under virtualization fan out into separate stack traversals between the guest and the backend. As a result, round-trip and context-switching overheads accumulate with the number of devices. In this letter, we identify that a dominant factor in this overhead lies not in the kernel-mediated I/O path alone, but in the per-device submission structure itself, which persists even in user-space, kernel-bypass storage frameworks such as SPDK. To address this, we propose cross-IP request coalescing, which relocates the fan-out point from the guest to the SPDK vhost-user backend. The guest submits multi-device I/O as a single compound request, and the accelerated bdev at the backend decomposes and dispatches it to each target device, replacing multiple per-device guest-backend round trips with a single submission. Evaluation in an SPDK-based virtualized environment shows that the proposed approach achieves up to 1.78x lower latency than the per-device baseline, with the benefit growing as concurrency increases.
To alleviate the overhead of moving massive data in modern data centers, Data Processing Units (DPUs) are increasingly deployed to offload NVMe-over-Fabrics (NVMeoF) remote storage access from the CPU. However, current heterogeneous CPU-DPU systems suffer from inflexible resource partitioning, leading to suboptimal utilization of computational power and network bandwidth. In this paper, we present DPIO, a unified I/O processing stack designed to harmonize the collaboration between CPU and DPU in NVMeoF environments. By implementing fine-grained task orchestration, DPIO effectively bridges the performance gap between DPU offloading and CPU processing, achieving near-optimal system performance across diverse workloads. Our real-world implementation demonstrates that DPIO delivers over 2 × higher throughput compared to conventional DPU-centric NVMeoF approaches, significantly enhancing the efficiency of data-intensive applications.
Wenhao Gu, Xuchao Xie, Yujuan Tan et al.· IEEE International Symposium...· 0 citations
Abstract —Today, Infrastructure-as-a-service providers are trying to minimize the cost of data center operations, while maintaining the Service Level Agreements. This can be achieved by one of the advanced state-of-the-art services of virtualization - the live migration capability. Live migration is defined as the process of transferring an active virtual machine from one physical machine to another without any disconnection. This is achieved by transferring all of the encapsulated states of the VM from one host to another. It has become an essential tool for efficient management of resources in a data center by enabling server consolidation and load balancing. There are two classical migration techniques, namely - pre-copy and post-copy, which employ different memory transfer mechanism during the offloading of a VM. In this paper, we propose a novel hybrid live migration technique by combining the existing pre-copy and post-copy approaches. Compare to its counterparts, our hybrid technique is a fast, efficient and a reliable migration technique.
P. Gangadhar· International Journal of Mac...· 0 citations
KV-cache offload is widely used to stretch GPU memory for LLM serving, but its storage behavior has not been characterized at the block-device level. In this paper, we study LMCache through realworld multi-session workloads that span same/different context $\times$ same/different prompt, using over 100 stateless requests per workload. Our comparison is observational rather than factorial: it contrasts two realizable deployment snapshots—LMCache 0.3.0 with buffered I/O and the kernel page cache, and LMCache 0.3.16 with prefix-aware deduplication and O_DIRECT. Key findings define the paper. First, the legacy stack appears almost read-free at the SSD layer in all four workloads, but a flush_ram experiment shows that this is conditional on page-cache warmth: once the cache is evicted, the same path issues about 1 GB of sequential reads in 683 ms. Second, deduplication delivers a dramatic write reduction only for the exact-repeat workload: one set collapses from 1.62 GB to 11.75 MB of writes, while the other three sets remain in the 2–5 GB range. Third, O_DIRECT converts a hidden and bimodal read cost into an explicit and stable one: on one set, warm TTFT rises from 27 ms to 93 ms because each of the 99 warm requests re-reads the same ~12 MB partial chunk from SSD. The main conclusion is therefore not “newer is better,” but a system trade-off: predictability versus average-case latency.
Ying He, Dingsen Shi, Yanbo Dai et al.· 2026 International Conferenc...· 0 citations
Spillway introduces a DPU-host hybrid data plane that repurposes idle host CPU resources to process spillover traffic when the DPU becomes the bottleneck, and decouples virtual switching capacity from static DPU hardware limits.
Xiaochong Jiang, Dian Fan, Yilong Lv et al.· Proceedings of the ACM SIGCO...· 0 citations
ParaFlowO is proposed, an architecture that Parallelizes processing elephant Flows across multiple CPU cores while preserving in-Order delivery and integrates a lightweight reordering mechanism to preserve packet order and controls parallelism to mitigate contention on shared state.
Changgang Zheng, Bowen Yang, Jin Ke et al.· Asia-Pacific Workshop on Net...· 1 citation
Virtual machine (VM) cross-cloud migration refers to the transfer of VMs between different cloud environments, enabling users to deploy their business with greater flexibility. However, VM cross-cloud migration faces many challenges in production due to the high latency and limited bandwidth of cross-cloud networks. These challenges are pronounced in disk migrations, where poor I/O performance persists and significantly constrains the migration process, even leads to migration failures. This drives the demand for VM migration drills, but existing solutions fail to effectively address it. We present Xtream, a production-level cross-cloud disk streaming migration system. By analyzing the business disk I/O patterns, we identify a macro-level locality pattern, characterized as multi-stripe disk access pattern, and develop a stripe-oriented prefetching algorithm in Xtream to improve I/O hit rate. Xtream designs a mechanism for collaborative cold and hot data migration to ensure timely responses to VM I/O requests. The evaluation demonstrates that Xtream reduces I/O latency by 88.2%, shortens cross-cloud VM startup time by 77.7%, and maintains an over 95% disk hit rate during migration, compared to the state-of-the-art streaming migration. In production, Xtream completes cross-cloud VM drills in 7–22 minutes depending on configuration, significantly reducing migration risks.
Tianlei Xiong, Yuchi Chen, Jiasen Li et al.· ACM Transactions on Architec...· 0 citations