A Unified Bandwidth Orchestration Framework for Hierarchical Data Storage Systems

Abstract

Hierarchical Data Storage Systems (HSSs) provide a cost-effective architecture that balances capacity and performance through internal data migration. Prior work has primarily focused on optimizing individual migration tasks, either within or across HSS tiers, or on exploiting device bandwidth to improve overall throughput. These approaches treat migration tasks in isolation, and the performance implications of executing heterogeneous migration tasks concurrently remain largely unexplored despite their prevalence in real-world HSS deployments. The growing adoption of Direct Data Access (DDA) architectures, in which accelerators access storage without CPU mediation, further amplifies this problem by removing a natural bandwidth arbiter from the I/O path. This paper presents an in-depth analysis of data migration behavior in commercial HSSs, uncovering substantial performance variability when multiple migration tasks execute concurrently. To mitigate this issue, we propose PASCAL, a system-level bandwidth orchestration framework that improves performance robustness in production-grade HSSs. Inspired by hydraulic systems, PASCAL adapts pressure/backpressure-style coordination to the multi-task migration setting: it treats each tier as a pressurized vessel and uses pressure gradients to allocate bandwidth across cache flush, tiering, garbage collection, and DDA flows. We evaluate PASCAL on a commercial OceanStor HSS across three hardware configurations and eight workloads spanning database, AI training, AI inference, and production traces. PASCAL achieves up to 20% higher throughput, 67% lower tail latency, and 79% reduced throughput jitter compared to local state-of-the-art controllers, while also stabilizing the performance jitter introduced by DDA architectures.