Filter Spectral Efficiency (FSE): Structured Network Pruning via Inter-Layer Spectral Coupling

Abstract

Structured filter pruning is an effective approach for reducing the computational cost of deep neural networks while preserving efficient deployment on standard hardware. However, most existing pruning methods evaluate filters independently, overlooking the inter-layer spectral coupling between consecutive layers, while many high-performing approaches rely on activation statistics or calibration data. We propose Filter Spectral Efficiency (FSE), a data-free structured pruning framework that estimates filter importance through the spectral coupling between adjacent layers. FSE constructs singular-value-based filter descriptors, builds inter-layer coupling matrices, derives a spectral pruning criterion, and allocates sparsity through a global adaptive budget. Extensive experiments on five CNN architectures for CIFAR-10 and ResNet-50 on ImageNet demonstrate that FSE consistently achieves a favorable accuracy-compression trade-off, outperforming representative structured pruning methods at comparable parameter and FLOPs reductions. On CIFAR-10, FSE reduces the FLOPs of ResNet-56 by 59.8% with only a 0.07 percentage-point accuracy drop, and prunes GoogLeNet by 70.7% FLOPs while improving Top-1 accuracy over HRank by 0.73 percentage points. On ImageNet, FSE improves the Top-1 accuracy of ResNet-50 by 1.14 percentage points over HRank at a comparable FLOPs reduction. These results demonstrate that inter-layer spectral coupling provides an effective and practical foundation for data-free structured network pruning.