2 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
Open-1B, a model trained under this regime, is released, together with its full pretraining dataset, every intermediate checkpoint, the training codebase, and the audit harness needed to reproduce and verify any step of its training.
A novel optimization method is proposed, NoLoCo, that does not explicitly synchronize all model parameters during training and does not require any collective communication, thereby reducing accelerator idle time and is more communication-efficient than DiLoCo.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
