Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping
This work suggests that "when to stop" should be treated as a training-time design choice, not merely an inference-time computation-allocation rule.
3 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This work suggests that "when to stop" should be treated as a training-time design choice, not merely an inference-time computation-allocation rule.
Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms. We analyze a single-loop multi-objective method combining a norm-constrained linear minimization oracle with entropic updates of task weights. For any finite-dimensional norm p...
Partially Informed Stochastic Optimization (PISO) is introduced, which combines the available gradient component with a running estimate of the residual to establish convergence guarantees that depend on the residual's variation and initial magnitude, without requiring a uniform bound on the unavailable gradient compon...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.