KItCAT: Knowledge Injection via Input Corruption for Auto-regressive Training
This work introduces KItCAT: Knowledge Injection via Corrupted Auto-regressive Training, a lightweight training strategy that reduces the need for paraphrasing in decoder-only LLMs and shows that KItCAT consistently improves over CPT across multiple datasets and model families.
Meghanadh Pulivarthi, Kushagra Bhushan, Vineet Kumar et al.
· 0 citations