#artificial intelligence #machine learning #cybersecurity Preprint

Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

Wei Zhai Xiang Liu Qiang Huang Rui Qian Le-Mao Liu Zi-Wei Li Zi-Qi Wang Zhi-Tao Huang De-Jing Dou

Oct 2026 · 0 citations · 26 references

Computer Science

TL;DR

By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework for LLM unlearning, and matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility.

Abstract

Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.

DOI arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

This work proposes Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score and provides an empirical path toward quantization-resilient unlearning.

Ravi Ranjan, O. Kotevska, Agoritsa Polyzou · 3 citations

Preprint Aug 2026

Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality

ADU is presented, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling, and achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks.

Xun-Lei Chen, Qi-Rui Ye, Yuang Li et al. · 0 citations

#artificial intelligence Preprint Sep 2026

Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection

With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean reference...

Jian-Wei Li, Jung-Eun Kim · 0 citations

#artificial intelligence Preprint Sep 2026

Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA

Reinforcement learning (RL)-based post-training has become an effective approach for eliciting reasoning capabilities in large language models (LLMs). However, adapting post-RL models to new knowledge domains or behaviors through subsequent supervised fine-tuning (SFT) can severely overwrite these capabilities. Existin...

Wen-Zhi Fang, Nicholas Tzou, L. Valkov et al. · 0 citations

#artificial intelligence Preprint Oct 2026

DNAlign: Dynamic Null-Space Safe Alignment for LLMs

DNAlign is a lightweight alignment framework that integrates control-theoretic optimization with null-space projection and restricts these perturbations to the harmful-related subspace derived from neutral hidden states, thereby preserving general knowledge and response quality.

Ji-Sheng Dang, Yu-Shu Zhao, De-Wei Liu et al. · 0 citations

Preprint Oct 2026

Sleeping Secrets: How Fine-Tuning Reawakens Privacy Risks in Language Models

This work proposes ReGap, a data-free attack that recovers private associations using task structure, filters them by answer-token likelihood, and updates the target model via low-rank adaptation, which requires neither target answers nor auxiliary genuine private supervision.

Jian-Hong Li, Jia-Hao Chen, Yuwen Pu et al. · 0 citations