This work unfolds the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm and treats the sparsity coefficient as a differentiable variable jointly learned with the network parameters and introduces a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed.
Abstract
Visual signals require compact yet sufficient representations for robust downstream prediction. Convolutional sparse coding (CSC) provides an explicit mechanism for suppressing redundant components while preserving signal content, but its sparsity coefficient is often manually selected during training. We propose a training-adaptive convolutional sparse coding framework for robust visual signal representation. Specifically, we unfold the CSC optimization with the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) and treat the sparsity coefficient as a differentiable variable jointly learned with the network parameters. From the information bottleneck perspective, this coefficient controls the trade-off between information retention and compression: the sparsity term promotes compact representations, while the reconstruction term together with task loss preserves task-relevant signal content. We further introduce a label-free post-training strategy that adjusts the compression strength for corrupted inputs with the main network parameters fixed. Experiments on CIFAR and ImageNet demonstrate competitive clean-data recognition and greatly improved robustness under different input perturbations.
Training-free sparse attention offers a practical acceleration solution to Diffusion Transformers (DiTs) via reducing computations without fine-tuning. It typically involves estimating the importance of query-key regions and deriving sparse masks to compute only the important candidates, which inevitably introduces app...
Cheng-Feng Han, Bao-Le Ai, Xian-Lu Bian et al.· 0 citations
Spatial-Aware Implicit Neural Representation (SA-INR), which enhances INRs by introducing local feature aggregation into the hidden representation space, and consistently improves reconstruction fidelity across different INR backbones and reconstruction tasks.
Chen Qing, Wen-Xin Zhang, Hao-Yu Wang et al.· Applied Sciences· 0 citations
Sparse PCA finds low-dimensional structure that loads on few features. Existing methods couple learning and feature selection by applying non-differentiable penalties that force retain-or-zero decisions during optimization. This eliminates features before the optimizer has established which ones matter, limits scalabil...
M. Aggarwal, V. Periwal· Transactions on machine lear...· 0 citations
Learned joint source-channel coding (JSCC) enables robust wireless image transmission by jointly optimizing the transmitter and receiver over differentiable channel models. For bandwidth-limited and time-varying visual links, a single model should support user-adjustable transmission rate and adapt to changing wireless...
A context-aware framework that specializes a pretrained semantic decoder to each group of pictures by optimizing only its channel-wise scale factors and transposed-convolution biases, enabling the receiver to reproduce the adapted decoder without full-model retraining is introduced.
P. Samarathunga, Yasith Ganearachchi, Thanuj Fernando et al.· IEEE Access· 0 citations
MTAR introduces multi-token prediction (MTP) to alleviate the sparsity and myopia of traditional NTP by imposing joint supervision on multiple future tokens; employs token-level contrastive regularization (TCR) to explicitly enhance the separability of sampled token representations and thereby improve representation di...
Guo Niu, Xiongfei Yao, Teng Wang et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 6, 2026
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
MIT News · Artificial Intelligence· news.mit.eduOct 28, 2026
Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.