Jia-Peng Chen

Poseidon: DAG-Guided Parallelism Search for LLM Pre-Training on Heterogeneous Clusters

Poseidon is an efficient and scalable LLM training framework designed with heterogeneity awareness, which employs two efficient, theoretically grounded strategies: stage-level pruning via early stopping with partial estimation, and layer-to-stage mapping exploiting a ridge-like distribution pattern.