Rethinking Omni Spatial-Frequency Representation for Efficient Face Super-Resolution

Abstract

Face Super-Resolution (FSR) faces a critical challenge in balancing reconstruction quality with computational efficiency when deployed on resource-constrained devices. While existing methods leverage CNNs or transformers, they are constrained by limited receptive fields or high computational complexity. To achieve both efficiency and high-quality results, we are motivated by spatial-frequency learning, centered on two pivotal designs: 1) the representation of dual domain features and 2) the fusion of dual domain information. In this paper, we propose OmniSF, an efficient FSR method that integrates spatial and frequency domains to enhance feature learning. Specifically, we design the omni spatial-frequency modulator (OSFM), which combines dual-domain token and channel mixers to effectively integrate the strengths of spatial and frequency domains. Furthermore, a dynamic spatial-frequency fuser (DSFF) is introduced to efficiently and sufficiently merge dual domain features, addressing the limitations of feature addition and cross-attention. Experiments demonstrate that OmniSF achieves state-of-the-art performance on benchmark datasets, with superior visual quality, a compact model size, and real-time inference speed.