CoCache: Accelerating Reads in KV Stores via Cooperative Metadata and Data Cache Management

Abstract

Modern LSM-based KV stores reduce read amplification with two in-memory caches, namely a table cache for metadata such as index and Bloom-filter blocks, and a data cache for value blocks. These caches draw from a shared memory budget and are jointly exercised on the read path, so the metadata-data split is inherently coupled and end-to-end read latency can be non-monotonic in the allocation. Allocating more memory to one cache may improve its hit rate but evict blocks from the other, yielding hard-to-predict performance especially under dynamic workloads. Most systems therefore rely on fixed, ratio-based heuristics, which can be far from optimal. We present CoCache, a cooperative cache-management framework that continuously tunes the metadata-data cache split. CoCache combines lightweight online hotness tracking with a unified latency model that captures the coupled impact of metadata and data caching on the read path. Using these signals, CoCache efficiently searches candidate splits and applies the one with the lowest predicted latency, then re-optimizes as access patterns shift. We implement CoCache in RocksDB and evaluate it on synthetic, benchmark, and production-derived workloads. Compared to state-of-the-art baselines, CoCache improves cache hit rates by up to 1.56 × , increases read throughput by up to 1.43 × , and reduces read latency by up to 31.2%.