We have 3 of 3 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Many widely used activation functions, including ReLU, GELU, Swish, and Sigmoid, produce predominantly positive outputs, resulting in a nonzero post-activation mean that we term activation bias. Despite its ubiquity, the effect of activation bias on learning dynamics remains poorly understood. Using controlled models,...
This work connects Adam loss spikes to both the mismatch between momentum timescales and finite-scale superquadratic loss geometry beyond the Hessian, and shows that confident cross-entropy losses develop a core--wall landscape comprising a narrow quadratic core followed by a steep wall, which produces effective superq...
It is shown that when a video model generates physically incorrect motion, the correct motion remains available inside the model and can still be made to control the generated video, and a stronger downstream write can restore physical motion, while excessive gain overshoots.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
