Preprint
Jul 2026
Agora: Collective and Permissionless Internet-Scale Pretraining of Large Language Models
Agora combines bandwidth-efficient pipeline-parallel model sharding over internet-grade links with multi-party, fault-tolerant collective operations, and enables collectively trained, collectively owned models, opening a path to open-source frontier training with economic sustainability.
Gil Avraham, Violetta Shevchenko, H. M. Dolatabadi et al.
· 0 citations