This work proves identifiability (exact recovery) up to trivial ambiguities for real analytic generating functions when source probability density functions have a finite number of discontinuities in the first derivative.
Abstract
Many machine learning systems try to explain complex data - like images or financial time series - in terms of hidden, independent factors that generated them. Recovering the true underlying factors, rather than some scrambled version of them, is the central challenge of nonlinear Independent Component Analysis (nICA). We prove identifiability (exact recovery) up to trivial ambiguities for real analytic generating functions when source probability density functions have a finite number of discontinuities in the first derivative. The Laplace distribution is the most prominent example satisfying this assumption. Our proof relies on the contrast between kinks in the source distribution and the smoothness of real analytic functions. Real analytic functions comprise a broad class of generating mechanisms, and can be approximated with Normalizing Flows or Variational Autoencoders with standard activation functions (e.g., tanh, softplus, GELU), so our result applies with minimal changes to existing training pipelines. We perform experiments on real and synthetic data with both Normalizing Flows and Variational Auto-Encoders demonstrating their identifiability properties. In experiments on CelebA data we recover several interpretable latent factors controlling unique attributes across the dataset.
We study which parameters of deep, unmasked, single-head attention are determined by its input--output function. For known positive nonconstant real-analytic normalizers, the function generically determines the effective scores and combined value map up to the signs induced by even normalizers. This proves the real-ana...
This paper analyzes the already established Linearly Scaled Hyperbolic Tangent function and proposes ReXTanh, a smooth-sparse hybrid function that provides C 1 continuity while remaining asymmetric, showing that keeping negative magnitudes can outmatch forced sparsity in harder recognition problems.
Emil Barciś, K. Rojek· Journal of Applied Mathemati...· 0 citations
Solving data-driven problems requires defining complex models and fitting them to data, neural networks being a motivating example. The fitting procedure can be seen as an optimization problem, which is often non-convex, and hence optimization guarantees are hard to derive. An opportunity is provided by viewing the mod...
Cristian Vega, C. Molinari, Lorenzo Rosasco et al.· 0 citations
This work proposes the local gradient neural operator (LGNO) as a lightweight and interpretable alternative for field temporal evolution prediction and source identification in typical mechanical problems and exhibits wide applicability to mechanical problems including diffusion, flow, and quantum phenomena.
Bai-Ming Zhang, Jin-Song Tang, Ying Xu et al.· 0 citations
Data and the governing partial differential equations (PDEs) of continuous-time dynamical systems provide rich structure for statistical learning. A central challenge is how to integrate data with PDE constraints in a principled way to improve generalization. In this paper, we characterize the statistical estimation an...
A statistical theory is developed that characterizes how regularization geometry and signal sparsity govern generalization near interpolation in the prototypical setting of high-dimensional regression and identifies regimes in which sparsity-promoting regularization makes exact interpolation much more accurate than app...
Gil Kur, Ileana Rugina, C. Dominé et al.· arXiv.org· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026