Li-Song He

ZOIF: Zeroth-Order Influence Function Estimation for LLM Data Selection Using Only Forward Passes

Influence functions enable principled data selection for large language models (e.g., identifying influential training samples for model editing, debugging, or efficient fine-tuning). However, scaling IFs to LLMs is hindered by the intractability of billion-scale Hessian matrices and the prohibitive memory overhead of...