On-Policy Attention Linearization
On-Policy Attention Linearization (OPAL) is introduced in which the hybrid attention student samples its own long-context trajectories and receives dense supervision from the frozen full-attention teacher.
We have 3 of 5 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
On-Policy Attention Linearization (OPAL) is introduced in which the hybrid attention student samples its own long-context trajectories and receives dense supervision from the frozen full-attention teacher.
For every odd integer $t\ge17$, we prove that an explicit circulant graph on $5t-10$ vertices is doubly saturated $R(3,t)$-good. The graph is triangle-free and has independence number $t-1$. Adding any nonedge creates a triangle, whereas deleting any edge creates an independent set of order $t$. This settles Conjecture...
A lightweight latency estimator is designed that simulates autoregressive token batch processing in the serving framework and estimates the time-to-first-token (TTFT) of queries and is incorporated into a latency-aware router that jointly optimizes latency, accuracy, and cost when assigning queries to model instances.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.