Skip to content
Preprint

What Does It Mean to Forget a Person? Individual-Level Unlearning in Vision-Language Models

Sep 2026 · 0 citations · 26 references
Computer Science

TL;DR

The proposed IDUnlearn-Bench is the first benchmark for individual-level multimodal unlearning in VLMs, which represents each individual as connected multimodal evidence and evaluates four task families: attribute access, identity access, identity binding, and identity reconstruction.

Abstract

Erasing individual identities from Vision-Language Models (VLMs) is uniquely challenging because personal data is entangled across modalities rather than stored as isolated attributes. However, existing multimodal unlearning benchmarks primarily evaluate attribute-centric forgetting, overlooking the more critical objective of individual-level unlearning: eliminating a model's ability to access, link, and reconstruct target-related information across modalities. To address this gap, we propose IDUnlearn-Bench, the first benchmark for individual-level multimodal unlearning in VLMs. It represents each individual as connected multimodal evidence and evaluates four task families: attribute access, identity access, identity binding, and identity reconstruction. Experiments on representative VLMs and unlearning methods show that successful attribute-centric forgetting often leaves substantial identity-level knowledge intact and can be non-monotonic: reducing one form of risk may amplify another. Models may suppress selected information while still identifying the target, linking records, or reconstructing the individual. These findings reveal a fundamental gap between forgetting information about a person and forgetting the person as a whole.

View source

Similar papers

#computer vision Preprint Aug 2026

AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning

AIM is proposed, a two-stage method that anchors an identity-forgetting target with a universal visual prompt and then matches the vision encoder to that target under a Fisher-based constraint, which achieves competitive identity forgetting while preserving non-deleted identities, prior knowledge, and visual perception...

Wonjun Lee, Jaehyuk Jang, Kangwook Ko et al. · 1 citation
#computer vision Preprint Aug 2026

Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models

PAVA pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone and gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.

Kangwook Ko, Jaehyuk Jang, Wonjun Lee et al. · 1 citation
#machine learning Preprint Oct 2026

SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning

The ability of vision-language models (VLMs) to associate visual identities with biographical information creates a need for selective unlearning of personally identifiable information (PII) while preserving permitted knowledge about the same individual. This setting is challenging because both sensitive and retained i...

Si-Qi Goh, Cap Dang Xuan Kiet, Tat-Jen Cham et al. · 0 citations
Preprint Sep 2026

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

ICM-Bench (Identity-Centric Memory Benchmark), which, to the best of the knowledge, is the first benchmark specifically designed to evaluate identity-centric reasoning over long video memories in multimodal agents, is introduced.

Shi-Du Ren, Yun-Ze Liu, Xingxian Liu et al. · 0 citations
Preprint Sep 2026

Not All Patches Are Equally Forgettable: Spatially Localized Domain Unlearning in Vision-Language Models

Pre-trained vision-language models (VLMs) exhibit strong cross-domain recognition performance even without additional training. However, this robustness can also preserve undesirable domain-specific behavior, as domain-related and semantic information often remain entangled within the learned representation space, maki...

Akanksha Singh, V. K. Kurmi · 0 citations
#machine learning Preprint Sep 2026

LLM Persona Unlearning

Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and goals. Post-training teaches conditional enactment and makes a helpful Assistant the default, but it does not erase alternative modes from the weights; explicit prompts can therefor...

Ke-Mou Li, Zhuan Shi, Qi-Zhou Wang et al. · 3 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.