Shamane Siriwardhana

Agora: Collective and Permissionless Internet-Scale Pretraining of Large Language Models

Agora combines bandwidth-efficient pipeline-parallel model sharding over internet-grade links with multi-party, fault-tolerant collective operations, and enables collectively trained, collectively owned models, opening a path to open-source frontier training with economic sustainability.