Skip to content
Log in
Preprint

DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning

Bin Huang Yifu Chen Zhilin Wang Yuanchao Du
Jul 2026 · 0 citations · 44 references
Computer Science

TL;DR

The Decoupled Representation Dynamic Network is proposed, which addresses two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate.

Abstract

Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our analyses suggest that classification supervision alone does not sufficiently preserve task-agnostic shared backbone representations over long incremental sequences. We identify two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate. We propose the Decoupled Representation Dynamic Network (DRDN), which addresses these challenges via two orthogonal mechanisms. For shared backbone representations, DRDN continuously applies masked image modeling (MIM) at every incremental step, with reconstruction gradients routed exclusively through the backbone, encouraging it to retain general visual structure beyond class-discriminative cues. For task-specific discrimination, DRDN employs hierarchical task token expansion across all transformer layers, with a modified per-task attention rule that reduces inter-task interference. We support this design with accuracy degradation analysis and cross-task confusion rate measurements. In the from-scratch ViT CIL setting (no external pretraining), DRDN consistently improves over strong token-expansion baselines with comparable backbone scale. On CIFAR100-B0 (10 steps), DRDN achieves 77.19% average accuracy, outperforming DKT by 1.36 points and DyTox by 3.53 points, with an advantage that grows at longer incremental sequences. Multi-seed validation confirms stability (+/-0.31%). The MIM decoder is active only during training, adding no inference-time parameters or computation.

arXiv Semantic Scholar
View source

Similar papers

Open access Jul 2026

Miles: Metric Learning With Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Miles decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion, and orchestrating an efficient expansion of the parameter space through guided optimization.

Kai Jiang, Zisong Lin, Hongyuan Zhang et al. · 0 citations
Preprint Aug 2026

TASSO: TAsk-Specific Subspace Optimization for Continual Learning of Vision-Language Models

TASSO, a new paradigm that efficiently preserves the latent space geometry while ensuring network plasticity, is introduced with two complementary techniques: subspace learning and geometry-aware knowledge distillation.

Changming Sun, Francesco Barbato, Matteo Caligiuri et al. · 0 citations
#machine learning Preprint Aug 2026

DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging

Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model's hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, and show two challenges absent in encoders: (1) the causal attention mask causes bias to accumulate across token positions, requiring position-dependent correction; and (2) not all token positions are equally important, i.e., high-entropy (decision-critical) positions matter far more than low-entropy ones. To address these challenges, we propose Decoder-Aware Representation Tuning via Surgery (DARTS). DARTS employs a novel entropy-weighted L1 loss to upweight correction at high-entropy positions where errors most affect generation quality, and a per-position additive bias that captures position-dependent error without overparameterization. We perform extensive evaluation on three domains: code generation (HumanEval), mathematical reasoning (GSM8K), and instruction following (AlpacaEval) on Llama-2-7B models, and show DARTS achieves significant improvement over the standard surgery approach while adding negligible parameters ($0.1\%$ of total parameters).

Aaryan Ajay Sharma, Sai Nishanth Padala, Seganrasan Subramanian · 0 citations
Preprint Aug 2026

Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models

A retraining-free VLM pruning framework called PORTA is introduced that derives a task- and modality-agnostic importance formulation based on activation variation, estimated from generic calibration data, which reliably captures feature-level representation utility across modalities.

Minseok Kang, Hyunwoo J. Kim, Chanyoung Kim et al. · 0 citations