DiCTR: A Distributed Lightweight Second-Order Optimization Algorithm for Deep Model Training
Distributed Cholesky triangular replacement (DiCTR) is proposed, which introduces a lightweight second-order method that employs efficient matrix factorization and sparse matrix replacement techniques to reduce system computation and memory overhead and designs a novel distributed training framework encompassing both d...