Skip to content

Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation

Sep 2026 · 0 citations · 26 references
Computer Science Mathematics

TL;DR

This work unfolds the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm and treats the sparsity coefficient as a differentiable variable jointly learned with the network parameters and introduces a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed.

Abstract

Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit mechanism for suppressing redundant components while preserving signal content, but its sparsity coefficient is often manually selected during training. We propose a training-adaptive convolutional sparse coding framework for robust visual signal representation. Specifically, we unfold the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) and treat the sparsity coefficient as a differentiable variable jointly learned with the network parameters. From the information bottleneck perspective, this coefficient controls the trade-off between information retention and compression: the sparsity term promotes compact representations, while the reconstruction term together with task loss preserves task-relevant signal content. We further introduce a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed. Experiments on CIFAR and ImageNet demonstrate competitive clean-data recognition and greatly improved robustness under different input perturbations.

View source

Similar papers

Preprint Oct 2026

iCATS: Fast Video Generation via Interaction-Aware Sparse Attention and Timestep-Adaptive Sparsity

Training-free sparse attention offers a practical acceleration solution to Diffusion Transformers (DiTs) via reducing computations without fine-tuning. It typically involves estimating the importance of query-key regions and deriving sparse masks to compute only the important candidates, which inevitably introduces app...

Cheng-Feng Han, Bao-Le Ai, Xian-Lu Bian et al. · 0 citations
Open access Sep 2026

Spatial-Aware Modulation for Implicit Neural Representations

Spatial-Aware Implicit Neural Representation (SA-INR), which enhances INRs by introducing local feature aggregation into the hidden representation space, and consistently improves reconstruction fidelity across different INR backbones and reconstruction tasks.

Chen Qing, Wen-Xin Zhang, Hao-Yu Wang et al. · 0 citations
Sep 2026

Scale-Invariant, Robust Sparse PCA for Large Data via Differentiable Penalties

Sparse PCA finds low-dimensional structure that loads on few features. Existing methods couple learning and feature selection by applying non-differentiable penalties that force retain-or-zero decisions during optimization. This eliminates features before the optimizer has established which ones matter, limits scalabil...

M. Aggarwal, V. Periwal · 0 citations
Preprint Aug 2026

Single-Model Adaptive Wireless Image Transmission via Feature Sparsity Regularization

Learned joint source-channel coding (JSCC) enables robust wireless image transmission by jointly optimizing the transmitter and receiver over differentiable channel models. For bandwidth-limited and time-varying visual links, a single model should support user-adjustable transmission rate and adapt to changing wireless...

Xianghao Cui, Li Lan, Qi He et al. · 0 citations
Open access 2026

Context-Aware Semantic Video Coding via Content-Adaptive Parameter Overfitting

A context-aware framework that specializes a pretrained semantic decoder to each group of pictures by optimizing only its channel-wise scale factors and transposed-convolution biases, enabling the receiver to reproduce the adapted decoder without full-model retraining is introduced.

P. Samarathunga, Yasith Ganearachchi, Thanuj Fernando et al. · 0 citations
Preprint Aug 2026

Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generation

MTAR introduces multi-token prediction (MTP) to alleviate the sparsity and myopia of traditional NTP by imposing joint supervision on multiple future tokens; employs token-level contrastive regularization (TCR) to explicitly enhance the separability of sampled token representations and thereby improve representation di...

Guo Niu, Xiongfei Yao, Teng Wang et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.