UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
Whether unlearning runs exhibit exploitable structure in weight space is investigated, and it is observed that models from different runs still lie in a shared evaluation-performance basin, suggesting that stronger models may be recovered through an unlearning-tailored soup strategy, reducing the need for repeated tuni...