This work investigates catastrophic forgetting in continual SSL from an explicitly geometric perspective, moving beyond downstream linear probe accuracy to characterize the evolution of the representation space directly and inform the development of novel methods that specifically preserve the embedding space geometry.
Catastrophic forgetting remains a fundamental obstacle to continual learning, where neural networks lose previously acquired knowledge while learning new tasks. Existing methods primarily mitigate forgetting through parameter regularization or experience replay, while the representation-space dynamics associated with forgetting remain less understood. We investigate latent representation evolution during sequential learning and introduce representation flux, a geometric measure of sample-level representation displacement across training. We show that representation flux is strongly associated with catastrophic forgetting across multiple benchmarks, with temporal analyses indicating that elevated flux can precede subsequent performance degradation. Representation displacement is also associated with confidence degradation, while complementary geometric properties provide additional information about sample-level forgetting. Motivated by these observations, we propose FlowLess-R, a representation-space regularization method that constrains replay representations relative to stored references while allowing continued learning. FlowLess-R is architecture-agnostic and integrates into replay-based methods through a representation-matching term. Experiments on SplitMNIST, SplitFashionMNIST, SplitCIFAR10, and SplitTinyImageNet show improved final average accuracy and reduced forgetting with ER, DER++, and ER-ACE. Our results identify representation flux as an informative geometric marker of forgetting and show that stabilizing latent representations provides a simple strategy for mitigating catastrophic forgetting.
TASSO, a new paradigm that efficiently preserves the latent space geometry while ensuring network plasticity, is introduced with two complementary techniques: subspace learning and geometry-aware knowledge distillation.
Changming Sun, Francesco Barbato, Matteo Caligiuri et al.· 0 citations
This paper studies what happens to the rest of the model when a class is forgotten, using a label-conditioned energy-based model (EBM) that assigns per-class energies, making the effect directly observable.
Syed Ali Ahmed, Syed Bilal Ahsan, Muhammad Zaigham Zaheer National University of Computer et al.· 0 citations
Traditionally, continual learning has assumed access to labeled data, yet many real-world applications -- such as lifelong robotics -- require models to adapt continuously from unlabeled streams. This has led to the development of continual self-supervised learning (CSSL), a rapidly growing area that lacks a dedicated, systematic review. In this work, we present a comprehensive survey of CSSL for vision, with connections to emerging vision-language settings. First, we analyze existing evaluation protocols and highlight inconsistencies that hinder fair comparison. We then examine why self-supervised objectives exhibit improved robustness to catastrophic forgetting, relating this to task-agnostic representations and smoother loss landscapes. Next, we organize existing methods into a unified taxonomy based on their forgetting-mitigation strategies, including distillation, replay, regularization, architectural approaches, model merging, and objective-level adaptation. Finally, we identify open challenges such as scalability and the need for fast adaptability. We argue that advancing CSSL requires moving beyond small-scale benchmarks towards continual pre-training paradigms for large-scale systems.
Sergi Masip Cabeza, Alicja Dobrzeniecka, Jonathan Swinnen et al.· 0 citations
In non-stationary streaming environments, simultaneously adapting to complex, non-linear domain shifts via continual learning while mitigating the catastrophic effects of severe, uncalibrated label noise poses a fundamental mathematical challenge. In this paper, we propose \FlatManifold{}, a novel, streamlined robust continual learning framework that utilizes a Nystr\"om manifold flattening map based on the kernel trick and projection onto an orthogonalized Reproducing Kernel Hilbert Space (RKHS). Unlike traditional methods that rely on complex, error-prone sample-filtering pipelines, the proposed approach exploits the intrinsic mathematical robustness of the flattened space itself. By mapping feature distributions onto a fixed orthogonal target topology with a ridge regularizer, the framework naturally smoothes and counteracts the influence of extreme label noise during the optimization process. Concurrently, catastrophic forgetting is prevented via a continual topology brake term that leverages the covariance matrix of past experiences. Extensive evaluation on real-world multi-session robotics datasets demonstrates that even under severe conditions featuring 40\% symmetric label noise, \FlatManifold{} successfully mitigates gradient corruption. Under extreme cross-session domain shifts spanning various seasons and lighting conditions, the proposed framework establishes high generalization capabilities, significantly outperforming standard sequential optimization baselines and proving that structural linearization itself serves as a powerful mathematical barrier against distributed label corruption.
Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.
Dezheng Han, Anlan Zhang, Zhiwu Zhu et al.· 0 citations