We have 3 of 14 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This work pretrain decoder-only models with approximately GPT-2 small and medium sizes on the FineWeb-Edu dataset and introduces Deep Delta Learning (DDL), which applies the delta rule over network depth.
The results suggest that fine-tuning is not just about how much a model changes, but how that change is spent, and that changing the accessible directions can qualitatively alter the outcome of fine-tuning.
This framework separates temporal alignment, plasticity, forgetting, and bounded rehearsal in recurrent sequence models, together with numerically stable positive-decay renormalization, to remain competitive in language modeling and improve length extrapolation on variable-digit addition.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
