WMDP++ is introduced, an extension of WMDP that addresses gaps in existing unlearning benchmarks by incorporating targeted extrac- tion of unlearned information and systematic evaluation on boundary questions.
Abstract
Recently, machine unlearning, the removal of specific training data influence from a model, has gained increasing attention. In large language models (LLMs), unlearning is particularly challenging due to the ambiguity of inputs and outputs. Con- sequently, rigorous evaluation is critical for assessing both safety and utility, and for driving progress in unlearning meth- ods. We identify two key shortcomings in existing unlearning benchmarks: (1) they do not actively test whether unlearned information can still be forcibly extracted, and (2) they fail to evaluate performance preservation on boundary questions, be- nign queries that are semantically close to the unlearned con- tent. Here we introduce WMDP++, an extension of WMDP that addresses these gaps by incorporating targeted extrac- tion of unlearned information and systematic evaluation on boundary questions. WMDP++ provides a more stringent and informative benchmark for evaluating unlearning in LLMs.
It is argued that effective unlearning must operate at the level of concepts, ensuring complete removal of unsafe applications while maintaining their correct and useful usage, thereby achieving conceptually meaningful and complete unlearning.
This work proposes ARIA (autoencoder-gated inference-time unlearning), a test-time unlearning method that leaves model weights intact and gates access to unwanted knowledge only when generation enters a forget-related state and introduces three post-unlearning adversarial attacks targeting weight-space and decoding-spa...
Ping-Zhi Li, Jinhao Duan, Vaishnav Tadiparthi et al.· 0 citations
J-Access is proposed, an inference-time audit that uses the Jacobian lens to map intermediate representations into vocabulary space and measures how often target concepts remain accessible along the model's output pathway, positioning J-Access as a model-level diagnostic for assessing residual susceptibility in unlearn...
Zirui Song, Hua-Xing Liu, Xiang Wang et al.· 1 citation
This work introduces Merging for Continual Unlearning (MCU), an approach that dynamically merges multiple one-shot unlearning adapters into a unified adapter upon receiving each new unlearning request and achieves superior unlearning effectiveness while preserving both retained knowledge and general multimodal utility.
Yuhang Wang, Lin-Lin Zhang, Haoxuan Ji et al.· 0 citations
Driven by stringent privacy regulations and data deletion requirements, machine unlearning has emerged as a critical field focused on selectively removing the influence of specific data from the pre-trained model. In this paper, we focus on achieving the unlearning objective while maintaining better model accuracy. Fir...
This work constructs a benchmark that captures unintended degradation on benign inputs sharing generic patterns with the forget set, and proposes Selective Protection with Anchored Regularization, which protects generic patterns via anchored activation filtering while reinforcing them through entity-abstracted enhancem...
Junxian You, Junkai Chen, Yuhao He et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 14, 2026
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
