We have 3 of 15 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
A derivative-free framework is developed that constructs Muon-style updates from structured finite differences from structured finite differences, and shows that random rank-one probing can reduce the number of function evaluations substantially, at the cost of less accurate updates.
Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by estimating update directions from loss evaluations, but existing methods require tuning a sensitive learning rate for each model and task. W...
It is shown that the contradiction in experience with neural surrogates in derivative-free optimisation dissolves once three factors are stated, and that these, rather than the fit accuracy a training curve reports, are what delimit when a learned local model pays.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
