Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network
This work relocating consensus from parameter space to function (output) space is provided, relocating consensus from parameter space to function (output) space: a KD event is a geometric contraction operator in logit space on the peers' predictive distributions, which is analysed in the Hilbert space of predictions on...
Lucas Qingyang Fang, Ti-Yao Liu, Jinhao Jing et al.
· 0 citations