Aug 2026· arXiv.org· Vol abs/2609.29711· 1 citation· 42 references
Computer Science
TL;DR
SPARK is proposed, a retention-correction decomposition that first freezes the learned post-task distribution and then applies selective correction around this stable reference and achieves effective selective PII suppression while preserving strong continual-learning utility and knowledge retention across diverse settings.
Abstract
Privacy-preserving continual learning (PPCL) must reduce the reproduction of sensitive content while retaining useful knowledge across sequential tasks. Formal privacy guarantees characterize randomized mechanisms, whereas operational output control concerns whether a trained model selectively reduces the likelihood of sensitive content in its outputs. In this work, we investigate the latter together with continual-learning utility under realistic task evolution. Retention and privacy correction operate at different granularities: task acquisition requires broad preservation of current- and old-task behavior, whereas privacy correction targets sparse annotated positions. Joint optimization leaves the current-task preservation target continually changing. We propose SPARK, a retention-correction decomposition that first freezes the learned post-task distribution and then applies selective correction around this stable reference. Self-Distillation Replay learns the current task while distilling behavior from previous tasks, and Post-Task Privacy Correction reduces annotated-PII likelihood while anchoring current- and old-task non-PII behavior to the resulting checkpoint. Extensive evaluations demonstrate that SPARK achieves effective selective PII suppression while preserving strong continual-learning utility and knowledge retention across diverse settings. Code and data will be released upon publication.
By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework for LLM unlearning, and matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility.
Wei Zhai, Xiang Liu, Qiang Huang et al.· Proceedings of the 2026 Conf...· 0 citations
This work proposes ReGap, a data-free attack that recovers private associations using task structure, filters them by answer-token likelihood, and updates the target model via low-rank adaptation, which requires neither target answers nor auxiliary genuine private supervision.
Jian-Hong Li, Jia-Hao Chen, Yuwen Pu et al.· 0 citations
Modeling LLM reasoning as search over a directed acyclic graph, a unified theoretical analysis of both the dynamics of post-training---OPSD and supervised fine-tuning (SFT) in continual learning---and the impact of pre-training on subsequent performance shows that reliable continual reasoning depends on how post-traini...
Controling how much and when teacher information is introduced improves specialisation while preserving broader capability is shown, and results show that controlling how much and when teacher information is introduced improves specialisation while preserving broader capability.
Ahmed Khaled Khamis, Xiao-Tong Ji, Hassan Jaber et al.· arXiv.org· 1 citation
This work investigates both gray-box and black-box threat models and proposes a unified attack framework, named OP-Print, based on the finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollo...
Qi Li, Xin-Chao Wang· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
MIT News · Artificial Intelligence· news.mit.eduSep 29, 2026
Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.