TrainSDC: Characterizing and Mitigating Silent Data Corruption in Large Language Model Training
This work presents the first systematic characterization of SDC vulnerability across major computation interfaces in both the forward and backward passes of Transformer training, and proposes TrainSDC, a characterization-guided protection framework consisting of Q/K-path recomputation, residual-gain monitoring, and exp...
Zhijie Xia, Haotian Xu, Si-Yu Yun et al.
· 0 citations