Speculative decoding accelerates inference for a large language model (LLM), referred to as the \emph{target model}, by first using a smaller model, referred to as the \emph{draft model}, to generate candidate tokens and then verifying them with the target model for acceptance or rejection. Prior studies primarily focu...
Yi-Chi Zhang, Zhi-Qi Wang, N. Gong et al.· 0 citations
Persistent memory helps LLM agents carry information across long interactions, but correct memory can still be used incorrectly when queries change or memory states evolve. Existing work mainly studies memory content errors or evaluates fixed test cases, leaving memory-use failures hard to discover systematically. We f...
Yu-Qiao Meng, Luo-Xi Tang, Ying-Xue Zhang et al.· 0 citations
This work identifies a class of user-issued instructions, Session Constraints, that are meant to constrain LLM's behavior for the remainder of a session but are silently dropped during compaction, and introduces COMPINT, an evaluation suite that evaluates compactors across three long-context scenarios: multi-turn chat,...
Zhiqi Wang, Yichi Zhang, Dongwon Lee et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.