Skip to content

Kinks vs. Smoothness: Identifiability of Real Analytic nICA for Laplace-like Sources

Sep 2026 · 0 citations · 29 references
Computer Science

TL;DR

This work proves identifiability (exact recovery) up to trivial ambiguities for real analytic generating functions when source probability density functions have a finite number of discontinuities in the first derivative.

Abstract

Many machine learning systems try to explain complex data - like images or financial time series - in terms of hidden, independent factors that generated them. Recovering the true underlying factors, rather than some scrambled version of them, is the central challenge of nonlinear Independent Component Analysis (nICA). We prove identifiability (exact recovery) up to trivial ambiguities for real analytic generating functions when source probability density functions have a finite number of discontinuities in the first derivative. The Laplace distribution is the most prominent example satisfying this assumption. Our proof relies on the contrast between kinks in the source distribution and the smoothness of real analytic functions. Real analytic functions comprise a broad class of generating mechanisms, and can be approximated with Normalizing Flows or Variational Autoencoders with standard activation functions (e.g., tanh, softplus, GELU), so our result applies with minimal changes to existing training pipelines. We perform experiments on real and synthetic data with both Normalizing Flows and Variational Auto-Encoders demonstrating their identifiability properties. In experiments on CelebA data we recover several interpretable latent factors controlling unique attributes across the dataset.

View source

Similar papers

#machine learning Preprint Oct 2026

The Identifiability and Observability of Deep Normalized Attention

We study which parameters of deep, unmasked, single-head attention are determined by its input--output function. For known positive nonconstant real-analytic normalizers, the function generically determines the effective scores and combined value map up to the signs induced by even normalizers. This proves the real-ana...

Pranav Venkata Konda · 0 citations
Open access Sep 2026

From symmetry to sparsity: A dual framework of LiSHT and ReXTanh activation functions for deep learning

This paper analyzes the already established Linearly Scaled Hyperbolic Tangent function and proposes ReXTanh, a smooth-sparse hybrid function that provides C 1 continuity while remaining asymmetric, showing that keeping negative magnitudes can outmatch forced sparsity in harder recognition problems.

Emil Barciś, K. Rojek · 0 citations
Preprint Oct 2026

Optimization by Reparametrization: Time-Warped Mirror Flows with Singular Geometry

Solving data-driven problems requires defining complex models and fitting them to data, neural networks being a motivating example. The fitting procedure can be seen as an optimization problem, which is often non-convex, and hence optimization guarantees are hard to derive. An opportunity is provided by viewing the mod...

Cristian Vega, C. Molinari, Lorenzo Rosasco et al. · 0 citations
#machine learning Preprint Sep 2026

Local gradient neural operator

This work proposes the local gradient neural operator (LGNO) as a lightweight and interpretable alternative for field temporal evolution prediction and source identification in typical mechanical problems and exhibits wide applicability to mechanical problems including diffusion, flow, and quantum phenomena.

Bai-Ming Zhang, Jin-Song Tang, Ying Xu et al. · 0 citations
#machine learning Preprint Oct 2026

Energy-Guided Learning for Heat Equation with Noisy Source: Optimal Estimation and Generalization Bounds

Data and the governing partial differential equations (PDEs) of continuous-time dynamical systems provide rich structure for statistical learning. A central challenge is how to integrate data with PDE constraints in a principled way to improve generalization. In this paper, we characterize the statistical estimation an...

Zi-Xian Zhou, Xiao-Hui Chen, Shang-Hua Teng · 0 citations

Grokking through the Lens of Minimum-Norm Interpolation

A statistical theory is developed that characterizes how regularization geometry and signal sparsity govern generalization near interpolation in the prototypical setting of high-dimensional regression and identifies regimes in which sparsity-promoting regularization makes exact interpolation much more accurate than app...

Gil Kur, Ileana Rugina, C. Dominé et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.