Aug 2026· Mathematics· Vol 14, pp. 2887· 0 citations· 26 references
TL;DR
An Embedded One-Class Classification (EOCC) framework for monitoring task-informed neural network representations andComparisons with depth-based, density-based, covariance-based, isolation-based, and end-to-end deep one-class methods show that EOCC is competitive and frequently achieves low Type II error while maintaining the nominal in-control acceptance probability.
Abstract
Artificial neural networks (ANNs) make predictions based on patterns learned during training; however, their reliability may deteriorate when the data distribution or the trained model changes. This paper proposes an Embedded One-Class Classification (EOCC) framework for monitoring task-informed neural network representations. The predictive network is first trained using its original classification or regression objective and then fixed. Embedded Support Vector Data Description (ESVDD) or Embedded Least Squares Support Vector Data Description (ELS-SVDD) is subsequently fitted to embeddings extracted from a selected hidden layer. The framework is evaluated through classification and regression simulations involving mean, covariance, and mixed distributional shifts. Additional experiments examine embedding layer and activation choices, direct neural network weight perturbations, and model changes induced by altered training conditions. Comparisons with depth-based, density-based, covariance-based, isolation-based, and end-to-end deep one-class methods show that EOCC is competitive and frequently achieves low Type II error while maintaining the nominal in-control acceptance probability, although no method is uniformly superior across all settings. Illustrative applications involving the Internet Firewall, ELEC2, and SINE1 datasets demonstrate how the framework can identify changes reflected in neural network representations. The present framework performs change detection only; adaptation and automatic model updating remain directions for future research.
Extensive experiments conducted on UCI and KEEL benchmark datasets demonstrate the superiority of the proposed IF-dRVFL and IF-edRVFL models over existing SOTA fuzzy and non-fuzzy approaches.
M. Sajid, A. Quadir, A. Rahaman et al.· 0 citations
This work proposes an NC-inspired training framework for simplifying deep networks during training, monitoring representation dynamics through the Inverse Fisher Criterion to identify both the split point between feature extraction and classification and the training stage at which simplification becomes viable.
Lorenzo Sciandra, Samuele Fonio, Roberto Esposito· 0 citations
Deep Belief Networks (DBNs) learn hierarchical generative models without class supervision. Here, we ask whether this purely unsupervised process nevertheless organizes internal representations according to the unknown data classes. We analyze successive layers of DBNs trained on MNIST, Fashion-MNIST, and KMNIST using the Generalized Discrimination Value (GDV), supervised probes applied only after training, a reconstruction-based measure of abstraction distance, effective dimensionality, and free sample generation. Remarkably, class-specific clustering generally increases with depth across datasets and network widths, although no label information is available during DBN training. Control experiments show that this effect depends on the learned feature structure and cannot be explained by random transformations, weight marginals, dimensionality reduction, or sigmoid saturation. The first hidden layers also frequently make class identity more accessible to linear and nonlinear probes. With greater depth, representations become increasingly compact and prototype-like as neurons acquire correlated feature directions. At the same time, GDV and probe accuracy reveal complementary aspects of class structure: improved average clustering can coexist with reduced accessibility for a few difficult class pairs. These findings demonstrate that layer-wise generative learning can spontaneously uncover and progressively amplify class-related structure in unlabeled data.
Patrick Krauss, Achim Schilling, Andreas K. Maier et al.· 0 citations
Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network and have recently been proposed as an alternative domain for downstream learning. While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters. Furthermore, the way discriminative information is distributed across INR weights remains poorly understood. We propose a hierarchical Mixture-of-Experts (HMoE) Transformer that processes INR weights using conditional computation aligned with the structure of the underlying implicit network. Coupled with a meta-learning framework that shapes INR parameters for downstream tasks, our model achieves state-of-the-art accuracy across standard benchmarks, ranging from low-resolution datasets to high-resolution ImageNet-1K. To gain insight into how INRs encode discriminative information, we develop weight-space attribution and pruning methods that identify parameters most relevant for classification. These analyses reveal how class-specific structure emerges within INR layers and support the suitability of MoE architectures for weight-space learning. Our approach advances both the performance and interpretability of weight-space classifiers.
Abstract.
The impressive performance of deep neural networks (DNNs) on a variety of learning tasks has spurred much investigation into improving their training and characterizing the trained functions. Recent work has shown the equivalence of DNNs in the infinite width limit and kernel machines relying on the neural tangent kernel (NTK) at initialization. These results suggest, and experimental evidence corroborates, that kernel machines relying on empirical kernels extracted from trained DNNs can act as surrogates for trained finite-width DNNs. The high computational cost of assembling the NTK, however, makes this approach infeasible in practice. In the current work, we study the performance of the conjugate kernel (CK), an efficient approximation to the NTK. For smooth function and logistic regression, we show that the CK performance is only marginally worse than that of the NTK and, in certain cases, much more superior. In particular, we establish bounds for the test losses, verify them with numerical tests, and identify the regularity of the kernel as the key determinant of performance. We also determine regimes where both kernel machines trained on features extracted from an underlying DNN are demonstrably superior to the latter and use this to suggest a recipe for accelerating DNN performance inexpensively. We present a demonstration of this on foundation models by comparing their performance on a classification task using a conventional technique and our prescription. We also show how our approach can be used to improve physics-informed operator network training as well as convolutional neural network training for vision classification tasks.
S. Qadeer, A. Engel, Amanda A. Howard et al.· SIAM Journal on Scientific C...· 0 citations
We propose a methodology based on the standard ReLU Deep Neural Networks (DNN) to make predictions and quantify their uncertainty. Classically, people rely on linear, non-linear, or non-parametric kernel methods to fit and then predict the time series. As the universal approximation ability was revealed for DNN, its application has become more and more popular for prediction tasks in various scientific areas. However, the corresponding uncertainty quantification has not been studied thoroughly. Particularly, the uncertainty in prediction will consist of two parts: (1) the future variability; (2) the estimation variability within training data. To capture both variabilities, we build the so-called pertinent prediction interval (PPI) with the DNN model estimator. We first explore the consistency property of the DNN estimator with beta-mixing dependent data. Subsequently, we show that the implied forward bootstrap series is still beta-mixing and possesses the same stationary distribution as the original time series in probability, which is a key condition to enable the PPI. Lastly, the desired PPI is built after imposing minimal conditions on the limiting distribution of predictive roots. Simulations and real-data analysis are deployed to challenge our approach with standard non-parametric methods.