We have 4 of 26 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions. We introduce **Latent Compaction**, which learns a compact document-specific cache through test-time training. We initialize cache entries u...
StateFlow is presented, a sequence pipeline parallelism system for models with linear recurrence that partitions each sequence into chunks and schedules their execution while propagating boundary states and gradients across chunks, thereby reducing activation lifetimes and improving training throughput.
QK-Restore consistently restores long-context capability at zero training cost while preserving reasoning performance; for instance, on HypeNet-5B it improves S3@256K from $65.4\%$ to $76.4\%$ while maintaining strong reasoning performance.
2D-RoPE is introduced, which organizes text into a 2D grid rather than a 1D sequence and assigns each token a row ID and a column ID, and suggests that viewing text in 2D can benefit language modeling.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
