Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space
This work proposes code surrogate gradient as the first order signal in deployable code space to acceleate optimization, and performs guided search to preserve deployment faithfulness in fine-tuning low-bit models across different quantization datatypes.
Shiguang Wu, Zhou-Chen Lin, Quan-Ming Yao
· 1 citation