Yun-qiang Li

Low-Rank Ternary Adaptation for Fine-Tuning Transformers

Ternary multiplicative adaptation is proposed, which represents discrete updates of ternary weights such as sign flips or zeroing through a low-rank Kronecker factorization into two small ternary matrices applied element-wise to ternary weights.