PC-SubMax: Efficient Prompt Compression via Regularized Submodular Maximization
While large language models (LLMs) are increasingly deployed in long-context scenarios, lengthy prompts can increase inference costs and latency and exacerbate the ``lost-in-the-middle''phenomenon. Selective prompt compression offers a model-agnostic approach to alleviating these issues. However, methods based on fixed...
Zi-Yi Zhang, Shuang Cui, Hao-Tian Zhang et al.
· 0 citations