LayerCheck: Adaptive Layer-Wise Checkpointing for Large Language Model Post-Training
LayerCheck is presented, a layer-wise adaptive checkpointing framework that selectively persists layers whose updates exceed a threshold that achieves up to 22.6× reduction in total checkpoint size and end-to-end training time compared to state-of-the-art systems, significantly lowering the cost of checkpointing.