Loop Dropout: Regularizing Shared Updates in Looped Language Models
Loop Dropout is introduced, which couples stochastic masking of adapter applications with inverse-survival rescaling to preserve expected update strength and promote effective adaptation across loops and outperforms existing LoRA variants and adapter regularizers.
Zi-Rui Zhu, Hai-Lun Xu, Xuan-Lei Zhao et al.
· 0 citations