Speculative decoding reduces sequential Target model calls by verifying multiple tokens from the Draft model in parallel. Yet KV Cache growth limits long-context serving under constrained GPU memory. Offloading KV to CPU memory relieves this pressure. However, existing offloading schemes restore the full KV history bef...
Fei Li, Song Liu, Shi-Qiang Nie et al.· 0 citations
OTDB, a key-value store tailored for overlapping track-based Disks is proposed and a Heterogeneous Multi-Granular Zoned Storage scheme that separates Sorted String Tables (SSTables) from different LSM-tree levels and maps them to zones with differentiated characteristics is developed, which reduces zone reset costs and...
Fang-Xing Yu, Zhi-Ke Li, Chi Zhang et al.· ACM Transactions on Architec...· 0 citations
A tenant pressure-aware performance isolation scheme, called IP-Zone, is proposed that partitions users into independent regions and maximizes zone parallelism through cross-resource allocation and flexible address mapping and improves IOPS and throughput.