Author

Zhi-Chao Cao

h-index 3 View on Semantic Scholar

We have 4 of 6 papers

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#natural language process... Preprint Oct 2026

Delta Tokenizer: Reference-Identical Tokenization of Session Deltas and Segment-Parallelism for Long Prompts on Multicore CPUs

An agent that drives an LLM sends the model the same conversation again and again, and each turn adds only a little text at the end. Inference engines reuse the KV cache of the shared prefix, so only the new tokens pass through the model. However, tokenization has no such shortcut. The CPU front end of the serving engi...

Zhen-Yu Zhang, Zhi-Chao Cao · 0 citations

Preprint Oct 2026

RiftANN: Efficient Graph Traversal for Vector Search with RDMA-Based Memory Disaggregation

Graph indexes for billion-scale vector collections can exceed a single server's DRAM capacity. Passive RDMA-based disaggregated memory provides scalable capacity without memory-node computation, but conventional best-first search performs poorly in this setting. Fine-grained and unnecessary remote reads increase commun...

Qi Lin, Zhen-Yu Zhang, Jun Kong et al. · 0 citations

2026

TokTier: Exact Stateful Tokenization for Agentic LLM Serving

LLM serving systems cache prompt KV state is serving systems cache prompt KV state, yet most front ends still re-tokenize the full request text on every call, leading to a mismatch that is most costly for coding agents.

Zhen-Yu Zhang, Zhi-Chao Cao · arXiv.org · 1 citation

Preprint Jul 2026

TokTier: Exact Stateful CPU+GPU Tokenization for Agentic LLM Serving

TokTier is a stateful CPU+GPU tokenization service for this two-mode workload, under one contract that re-tokenizes a small window around the append and splices only when a per-request check finds a stable pre-tokenization boundary.

Zhen-Yu Zhang, Zhi-Chao Cao · 3 citations