LinCa decomposes cached features into sub-components with distinct continuity properties via a lightweight invertible network and applies differentiated prediction orders matched to each component, forming a unified Decompose-Predict-Reconstruct pipeline.
Abstract
Diffusion models have achieved remarkable success in image and video generation, yet the high computational cost of iterative sampling remains a critical bottleneck for practical deployment. Feature caching has emerged as a promising acceleration paradigm by reusing or predicting intermediate features across timesteps. However, existing training-free methods apply uniform prediction strategies that cannot adapt to the heterogeneous feature dynamics, causing significant quality degradation under high acceleration ratios. We propose LinCa, a feature caching framework based on learnable invertible networks. LinCa decomposes cached features into sub-components with distinct continuity properties via a lightweight invertible network and applies differentiated prediction orders matched to each component. The strict invertibility guarantees lossless reconstruction back to the original feature space, forming a unified Decompose-Predict-Reconstruct pipeline. By training separate predictors for different models and timestep segments, LinCa adapts to heterogeneous feature dynamics. Experiments on FLUX, Qwen-Image, and HunyuanVideo demonstrate that LinCa, with less than 0.2% additional parameters, significantly outperforms existing methods and maintains near-lossless quality at 5-7x speedup. Code: https://github.com/QHR69/LinCa
OmniCache is proposed, a unified hierarchical caching framework that performs multidimensional feature reuse through Token Cache, Frame Cache, Block Cache, and Layered Cache that reuses spatial features in temporal layers and temporal features in spatial layers, while Layered Cache captures cross-step redundancy at the model-layer level.
The exceptional generative capabilities of modern diffusion models are fundamentally bottlenecked by the quadratic computational complexity of their attention mechanisms. While recent feature caching strategies attempt to accelerate inference by skipping layers at static intervals, they fail to account for the non-linear evolution of latent features, inevitably causing severe structural distortions and temporal flickering. To address this, we propose AST-ToMe (Adaptive Step-Aware Thresholding), a novel dynamic gating mechanism that utilizes a runtime L2 norm feature drift metric to adaptively determine whether to compute or reuse attention states. Furthermore, we extend AST-ToMe to video stream generation through a cross-frame state inheritance design. Experimental results demonstrate that for single-image synthesis, AST-ToMe achieves a 15.3% reduction in inference latency with near-lossless perceptual quality (LPIPS: 0.0023). In continuous video generation, our method not only accelerates inference but also serves as a robust temporal anchor. By effectively suppressing random stochastic variations, AST-ToMe successfully reduces Temporal Jitter from 0.2059 to 0.1865, paving a highly efficient path for stable, flicker-free video generation.
Shuzhi Zheng· International Conference on...· 0 citations
Diffusion models achieve strong performance in generative modeling but incur high memory and computational costs due to large model parameters and iterative denoising procedures, limiting deployment in resource-constrained environments. This paper presents a unified framework to improving inference latency and memory efficiency of diffusion models by jointly optimizing activation reuse and weight storage. We introduce a representation-aware caching strategy(ReCaT) which reduces redundant computation during inference by selectively reusing intermediate activations along the diffusion trajectory. ReCaT identifies effective caching points based on representational similarity measured using centered kernel alignment(CKA), enabling informed activation reuse without degrading generative quality. To complement activation reuse, we propose Adaptive Clusteringguided Quantization with Entropy Encoding (ACQ-E) for efficient weight storage. ACQ-E uses histogram-guided clustering to model weights, followed by adaptive mixed-precision quantization and entropy encoding to reduce storage and cost. The proposed framework integrates representation-aware caching and adaptive quantization into a unified optimization pipeline, leading to reduced effective bit-width, smaller model size, and lower computational cost while maintaining generative fidelity. Experiments on standard benchmarks, including LSUN Churches, LSUN Bedrooms, CelebA-HQ, MS-COCO, and PartiPrompt, demonstrate efficiency gains with minimal impact on image quality when compared to full-precision.
Deepanshi Sharma, Rudra Pratap, Shivam Jaiswal et al.· International Conference on...· 0 citations
MobileWan becomes the first 5B-scale video diffusion model deployable on a commercial mobile device and proposes a learnable attention head pruning method based on binary per-head gates optimized end-to-end using a noise-biased sparsity objective and distillation-based finetuning.
Mohsen Ghafoorian, Denis Korzhenkov, Adil Karjauv et al.· 1 citation
Recent advances in generative AI have significantly increased the demand for high-resolution image and video generation, positioning diffusion models as a core technology. Among them, Diffusion Transformers (DiTs) have emerged as the state-of-the-art (SOTA) models due to their scalability and output quality. However, self-attention in DiTs incurs significant computational overhead, leading to excessively long latency as the complexity grows with the fourth power of the output resolution. While prior works have attempted to mitigate this cost using sparsity and quantization techniques, they fall short of effectively reducing the computational cost in high-resolution DiTs. In this paper, we present APT, a software-hardware co-designed accelerator for high-resolution DiTs. APT leverages attention probabilities as a unified importance metric to jointly optimize computation through fine-grained pruning and adaptive precision scaling. At the algorithm level, we propose Attention Probability-guided Adaptive Dual Thresholding (APDT), which dynamically performs element selection and precision assignment using dual thresholds. To ensure compatibility with memory-efficient FlashAttention, we introduce Timestep-Aware FlashAttention (TAFA), which predicts attention probabilities across timesteps by exploiting temporal similarity. At the architecture level, we co-design a specialized accelerator that efficiently supports irregular sparsity and dual-precision execution, featuring dynamic mask management, address translation, dual-precision compute units, and a tile-based dataflow. Finally, we evaluate APT on SOTA DiT models, including PixArt-$\alpha$, Stable Diffusion 3, and FLUX. APT achieves up to 8.16$\times$ speedup and 14.98$\times$ higher energy efficiency over NVIDIA A100, and up to 3.01$\times$ speedup and 2.04$\times$ higher energy efficiency over EXION, a SOTA diffusion model accelerator.
Sungyeob Yoo, Seeyeon Kim, Joonyong Park et al.· 0 citations
ACID is a lightweight, training-free wrapper that monitors the rate of change of each method's existing drift signal to dynamically switch between a low and a high threshold and consistently expands the Pareto frontier of visual quality versus inference speed beyond what any fixed threshold achieves.
Om Agrawal, Saurabh Agarwal, Aditya Akella· 0 citations