Skip to content

Spectral Saliency for Machine Unlearning

Aug 2026 · 0 citations · 57 references
Computer Science

TL;DR

Inspired by Muon, this work adopts the spectral view for unlearning and proposes Spectral Saliency Unlearning (SSU), a thresholding approach that thresholds weak singular components and updates only those directions supported by a confident unlearning signal.

Abstract

Machine unlearning (MU) aims to remove the influence of specific training data while preserving model utility. As the name suggests, MU can be viewed as the inverse of learning, using gradient-based updates to reduce the influence of a forget-set by counteracting the previously learned behavior. Recently, Muon, a gradient descent variant, has been introduced. Muon applies spectral magnitude normalization to encourage exploration of rare directions and demonstrates promising performance. Inspired by Muon, we adopt the spectral view for unlearning and propose Spectral Saliency Unlearning (SSU). SSU thresholds weak singular components and updates only those directions supported by a confident unlearning signal. We further provide theoretical justification for this thresholding approach from the perspective of the forgetting-retention trade-off. Experiments across image classifiers, diffusion models, and LLMs demonstrate SSU's effectiveness.

View source

Similar papers

2026

DeepU: Deeper Granular Within-Layer Machine Unlearning

Machine unlearning (MU) aims to remove the influence of selected data from trained models, offering an efficient alternative to full retraining. With the rise of increasingly stringent privacy regulations, including the right to be forgotten, machine learning models must incorporate mechanisms that ensure compliance wh...

Anudeep Vurity, Zhi-Sheng Yan, Massimiliano Albanese · 0 citations
#artificial intelligence Preprint Sep 2026

Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders

This work introduces SCALPEL, a contrastive sparse autoencoder designed to learn more selective forget features and shows theoretically that contrastive training promotes target-selective features and that the selection score controls expected background knowledge perturbation.

Itai Zehavi, Fanny Jourdan, Ulrich Aivodji · 0 citations
Preprint Aug 2026

Cross-Domain Generalization in Machine Unlearning via Label-Conditioned Energy Magnitude Regularization

This paper studies what happens to the rest of the model when a class is forgotten, using a label-conditioned energy-based model (EBM) that assigns per-class energies, making the effect directly observable.

Syed Ali Ahmed, Syed Bilal Ahsan, Muhammad Zaigham Zaheer National University of Computer et al. · 0 citations
#artificial intelligence Preprint Sep 2026

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happen...

Hao-Ran Tang, Andrew Tan, Rajiv Khanna · 0 citations
#artificial intelligence Preprint Oct 2026

Local Support Learning

This work proposes Local Support Learning (LSL), a general-purpose framework that augments gradient-based training for retention of prior capabilities without access to prior data, and presents a gate based on a Gaussian Mixture Model, whose likelihood decays rapidly away from its training data, giving it a natural ten...

Assaf Ben-Kish, Akarsh Kumar, James R. Glass et al. · 0 citations
#artificial intelligence Preprint Aug 2026

Forget or Fine-tune? A Comparative Study of Machine Unlearning Strategies for Noisy Label Correction

This work conducts a comparative empirical study of five MU methods across symmetric, asymmetric, instance-dependent, and open-set noise on CIFAR-10, CIFAR-100, and the real-world noisy dataset Food-101N and finds that the appropriate unlearning strategy is conditioned on the noise structure.

J. L. Sant'Ana, Filipe R. Cordeiro · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.