This paper forms CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors and proposes a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula.
Abstract
Convolutional Neural Networks (CNNs) have undoubtedly revolutionized image data analysis and the field of computer vision. As the cornerstone of CNNs, the convolution operation enables the networks to extract abstract features and uncover hidden relationships in the image data. This paper considers the problem of estimating convolution filters from a statistical perspective using a classical tool --- Stein's formula. We first formulate CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors. Furthermore, we propose a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula. Theoretical analysis suggests that our estimation achieves an optimal convergence rate, comparable to that of generalized linear models where the link function is known. Extensive simulation studies and real data analyses demonstrate that our approach outperforms popular deep learning algorithms, such as Adam. Notably, our method extends beyond filter estimation and can be applied to nonlinear dimension reduction, providing a viable pathway for representation learning.
Dynamic convolution enhances convolutional neural networks (CNNs) by adapting kernels to input content, but it expresses the effective kernel as a linear mixture of a small number of basis kernels, which limits expressivity and complicates optimization as the mixture size grows. In this work, we revisit dynamic convolu...
A novel approach to CNN architecture optimization is proposed by leveraging mathematical series—such as arithmetic, geometric, or harmonic progressions—to define structured, scalable, and interpretable layer configurations, offering a lightweight, reproducible framework for CNN configuration that can be adapted across...
K. AshaRani, S. Gowrishankar· Neural computing & applicati...· 0 citations
This paper studies the cryptanalytic extraction of convolutional neural networks (CNNs). Existing cryptanalytic extraction attacks on CNNs assume that the network architecture is known, and try to recover model parameters.In this paper, we prove for the first time that the architecture assumption can be removed for CNN...
Jia-Shuo Liu, Rui-Jie Ma, Man-Man Li et al.· 0 citations
The hyperbolic Res-MLP (HR-MLP), an architecture extending the attention-free MLP to a non-Euclidean space and enabling direct hyperbolic operations with fewer parameters, facilitating faster training and superior performance compared to Euclidean counterparts is introduced.
A correspondence is presented linking discrete filter symmetry and the relativistic energy--momentum relation under the widely used nonlinear rectified convolution operation.
The demand for more accurate Convolutional Neural Networks (CNNs) has led to increasingly deep and computationally expensive models. These architectures face limiting deployment and resource constraints when implemented. Static CNN models treat all input samples with identical computational density. These models are in...
Joe Kareh, Rodrigue Imad· Applied Informatics· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.