PERK: Long-Context Reasoning as Test-Time Learning
Long-context reasoning requires accurately identifying relevant information in extensive, noisy input contexts. In this work, we propose PERK (Parameter Efficient Reasoning over Knowledge), a scalable approach for learning to encode long contexts using gradient updates at test time. Specifically, PERK employs two neste...
Zeming Chen, Angelika Romanou, Gail Weiss et al.
· 6 citations