ProxyFormer: A Dual-Stream Proxy Architecture for Ultra-Long Context and High-Resolution Generation
This work proposes ProxyFormer, a general dual-stream architecture built upon proxy tokens, and introduces factorized multi-level compression/decompression, layer-wise dynamic compression ratios, asymmetric dual embeddings, and a proxy-only KV-cache inference scheme.
Zhongpan Tang
· 0 citations