Aug 2026· Proceedings of the ACM SIGCOMM 2026 Conference· pp. 2033-2039· 0 citations· 20 references
TL;DR
Feather is presented, a system that reduces this overhead by compressing intermediate activations before transmission and reconstructing them before downstream layers resume execution, consistently outperforming existing compression schemes, including linear autoencoder, PCA, and SVD.
Abstract
Large transformer models are increasingly deployed across geographically distributed GPU clusters due to capacity, cost, and locality constraints. When inference is partitioned across sites, intermediate activations must be transmitted over wide area network (WAN) links at each partition boundary, introducing significant communication overhead. We present Feather, a system that reduces this overhead by compressing intermediate activations before transmission and reconstructing them before downstream layers resume execution. Feather learns a compact representation of activation tensors using a lightweight neural codec trained with a reconstruction objective while keeping the original model frozen. Across encoder and decoder transformer models, Feather achieves up to 48× activation compression while maintaining accuracy close to the baseline model. Under representative WAN conditions (e.g., 10 Gbps bandwidth and a 10 ms RTT), this reduction yields up to 4.96× improvement in end-to-end latency, consistently outperforming existing compression schemes, including linear autoencoder, PCA, and SVD.
AE-PSL is proposed, a communication-efficient PSL framework that compresses intermediate activations and gradients using a lightweight AutoEncoder placed at the split layer and introduces a novel two-stage alignment mechanism, which adapts the AE to the pre-trained model's feature manifold and client-specific feature distributions before DFT.
Bas Meuwissen, Vasileios Tsouvalas, N. Meratnia· 1 citation
Fine-grained weight pruning and activation sparsification have emerged as effective approaches for reducing the compute and memory cost of inference for Transformer models. In the moderate-sparsity regime, Gustavson's dataflow provides a natural execution model for exploiting both activation and weight sparsity on vector processors through metadata-driven indexed accumulation. However, existing RVV architectures lack native support for this pattern, forcing kernels to rely on software index decoding and L1-backed indexed memory operations that keep sparse tensor contractions far below their roofline performance bound. We present Ventaglio, a runtime-configurable sparse execution unit coupled with RVV ISA extensions that drives sparse tensor contractions toward their roofline through indexed gather-accumulate-scatter support. Integrated into an open-source vector processing cluster and implemented in 12nm FinFET, Ventaglio accelerates sparse tensor contraction kernels by $6.9\text{--}7.4\times$ over optimized RVV baselines, with only $3.1\%$ area overhead for a cluster of tightly-L1 coupled vector processing elements. We build a performance-accurate instruction-level model of the Ventaglio extension, calibrate it against RTL implementation, and leverage it for scale-out performance analysis on a large $4\times4$ multi-cluster system. Using a DuoGPT-pruned LLaMA-3-8B model with practical $40\text{--}60\%$ dual sparsity, Ventaglio achieves $2.40\text{--}5.25\times$ and $2.06\text{--}3.16\times$ speedup over dense baselines during prefill and autoregressive decoding, respectively.
Bowen Wang, Chi Zhang, Diyou Shen et al.· 0 citations
Large-scale scientific simulations generate volumetric data at rates that far outpace advances in storage and network bandwidth, making effective lossy compression increasingly critical. However, conventional compressors often struggle to preserve fine structural details at high compression ratios (CRs), and implicit neural representations (INRs) require costly per-volume optimization and produce models with fixed CRs. To respond, we present EVOLVE, an autoencoder (AE)-based volume-compression framework that targets high CRs for offline compression, with three key contributions. First, we construct a large-scale cross-domain database of 6,376 volumes from 21 scientific simulations, curated via perceptual hashing to ensure diversity, enabling the optimized model to extract features that generalize across volumes within the covered scientific simulation domains. Second, we reexamine the design space of AE-based compressors and incorporate several macro- and micro-designs into a vanilla AE to develop EVOLVE, which substantially improves the expressive power and compression capability. Third, we develop a learnable gain mechanism with a three-stage training strategy to enable variable-rate encoding, allowing a single model to support continuous CR adjustment at inference time. Experiments on multiple unseen scientific simulation datasets demonstrate that EVOLVE achieves substantially higher CRs than conventional compressors at comparable reconstruction quality, while delivering compression speeds that are orders of magnitude faster than INR-based methods, highlighting its promise as a strong alternative for compressing scientific data. The code, model weights, and results are available on our project page at https://evolve-vis.github.io.
Chunked Muon (CMuon) is introduced, a simple yet highly effective strategy that partitions these matrices into independent sub-components prior to orthogonalization, effectively overcoming the late-stage convergence plateaus of vanilla Muon.
Distributed deep neural network (DNN) training across multiple workers is widely adopted to accelerate the training of large models and datasets. However, their performance is often limited by the synchronization of inter-worker data (i.e., gradient tensors). Top-k sparsification serves as the de facto compression approach to alleviate the communication bottleneck by retaining only the k largest-magnitude gradient elements before transmission to other nodes. However, traditional top-k sparsification has two key limitations. First, gradients at each DNN layer are represented as multi-dimensional tensors, with the selected k elements typically concentrated in a subset of dimensions. This leads to many dimensions being missed during training, and hence poor convergence. Second, traditional top-k sparsification relies on the global sorting of all gradient elements per layer, leading to underutilized GPU core parallelism and low training throughput. To this end, we propose ADTopk, an all-dimension top-k sparsification scheme that selects the k largest elements across all dimensions of the per-layer gradient tensor, ensuring that every dimension contributes at least one element and eliminating dimension missing. ADTopk also enables independent local sorting within each dimension, allowing parallel execution across dimensions to enhance GPU utilization. Furthermore, we enhance ADTopk with system-level optimizations: (i) interleaved sparsification to accelerate convergence, (ii) partial sparsification to reduce sparsification overhead, and (iii) hybrid collective communication to improve sparse communication efficiency. Experiments on local and cloud clusters show that ADTopk improves convergence accuracy by 2.75%-22.31% and achieves 20.8%-268.0% higher training throughput than state-of-the-art sparsification approaches.
Zhangqiang Ming, Yuchong Hu, Patrick P. C. Lee et al.· ACM Transactions on Architec...· 0 citations
Deploying high-accuracy neural networks on resource-constrained edge devices remains challenging, as existing approaches treat training, compression, and hardware synthesis as separate stages, leaving a gap between software-trained models and efficient end-to-end deployment with limited support for interpretability. We propose Bern2Edge, an end-to-end framework that uses knowledge distillation to convert a pretrained teacher feed-forward network into hardware-efficient representations via Bernstein polynomial activations. This representation enables two deployment paths: (i) a high-fidelity LUT-based realization that preserves model fidelity under compression, and (ii) a symbolic rule-based representation derived from Bernstein activation geometry, enabling interpretable inference with explicit input-space constraints. The resulting BNNs achieve up to 2.12 percentage-point (pp) accuracy improvement over ReLU under identical compression constraints. At the system level, Bern2Edge achieves up to 99.8% latency reduction and 95.2% BRAM reduction relative to a W8A8 quantized teacher on an AMD Xilinx KV260 FPGA, while maintaining accuracy within 0.5 pp, and further deploys on a low-power Spartan-7 XC7S15 FPGA. The rule-based path reduces DSP usage by up to 89.0% at a cost of 1.5 pp in total accuracy.