FlexComp: One Model for Every Ratio in Context Compression
Soft context compression condenses a context into a few memory tokens that a frozen LLM consumes in place of the raw text, but existing compressors fix the compression ratio at training and inference: each deployed ratio requires a separately trained model, and the chosen ratio is applied uniformly to all inputs, whose...
Kai-Yan Zhao, Zhong-Tao Miao, Akiko Aizawa et al.
· 1 citation