Debias-SparseGPT: Bias-Aware Pruning for Large Language Models
Debias-SparseGPT is introduced, a post-training pruning method incorporating representational debiasing using a second-order term defined over demographically contrasting inputs that consistently reduces pruning-induced bias compared to SparseGPT while preserving model perplexity and zero-shot accuracy.
Irina Proskurina, Guillaume Metzler, Antoine Gourru et al.
· 0 citations