An efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format and observing that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed is observed.
Abstract
Abstract.
This work proposes an efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format. The two settings we consider are variational inference and density estimation, whereby one has access to either the unnormalized analytic formula of the distribution or the samples generated from the distribution. In particular, the compression is done through a two-stage approach. In the first stage, we use existing subroutines to encode the distribution function in a tensor train format. In the second stage, we use an NTT ansatz to fit the obtained tensor train. For the NTT fitting procedure, we use a log barrier term to ensure the positivity of each tensor component and then utilize a second-order alternating minimization scheme to accelerate convergence. In practice, we observe that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed. Through challenging numerical experiments, we show that our approach can accurately compress target distribution functions.
Reproducibility of computational results. This paper has been awarded the “SIAM Reproducibility Badge: Code and Data Available” as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/Xun-Tang123/Non_Negative_Tensor_Train and in the supplementary materials ( Non_Negative_Tensor_Train-main.zip [51.4MB]), linked from the main article webpage. [Formula: see text]
This work presents a method to accelerate the optimization of learning high dimensional functions using deep neural network (DNN), and studies the effect of adding features which distill pretrained DNN into TNs using a discretize and decompose strategy.
This paper proposes a novel Tensor Train (TT)-based tensor-on-tensor regression optimization framework for variable selection based on mode-1 hyperslice sparsity. The formulation incorporates an l2,0-regularized term on the first TT-core while imposing Stiefel manifold constraints on the remaining M−1 TT-cores. Leveraging the property that the group sparsity of the first core is equivalent to the hyperslice sparsity of the global structure, we establish theoretical guarantees for the uniform variable-selection consistency of the proposed model. To efficiently solve the proposed model, we design an alternating iterative algorithm equipped with a preconditioned metric and prove its convergence to a critical point. Extensive numerical experiments on both synthetic and real-world datasets demonstrate that the numerical solutions generated by our algorithm exhibit exact support recovery in practice, tightly aligning with our theoretical analysis.
This work introduces a novel and efficient solver for the underlying HJB equation based on the functional tensor train (FTT) format, which leverages latent low-rank structures to efficiently approximate high-dimensional functions, enabling both model compression and rapid computation.
Robert Gruhlke, Julius Berner, David Sommer et al.· 0 citations
This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
This work provides a theoretical analysis which establishes a non-asymptotic reconstruction error bound that characterizes the effects of sampling complexity, optimization convergence, and model mismatch in a structured tensor approximation problem.
Jeongmin Chae, Usama Saleem, Selin Bac et al.· 0 citations
We develop a tensor reduced-order modeling (TROM) framework for optimization-based inverse problems governed by parameter-dependent dynamical systems. The approach approximates the parameter-to-observation map directly in tensor-train format, using either TT-SVD or TT-Cross compression, and integrates the resulting representation into a regularized nonlinear least-squares formulation. Beyond accelerating forward evaluations, the low-rank tensor structure is used to reformulate the inverse problem in reduced coordinates, assemble the Gauss--Newton quantities without forming the full observation-space Jacobian, and perform TROM-based objective minimization over the discrete parameter grid. This tensor optimization step can be used either as a stand-alone approximate minimization procedure or as a data-informed initialization for a subsequent Gauss--Newton solve. The method is studied for two inverse problems: an inverse heat-transfer problem in a heterogeneous medium, where the unknown parameters describe the locations of multiple low-conductivity inclusions, and a FitzHugh--Nagumo parameter-estimation problem with a highly nonconvex optimization landscape. Numerical experiments assess the effects of ROM approximation error, measurement noise, regularization, initialization, spatial discretization, and increasing parameter dimension. The results show that TROM can reproduce the behavior of full-order inversion at a substantially reduced online cost. The experiments also demonstrate that reduced-coordinate inversion, tensor-based optimization, and appropriate regularization improve robustness in higher-dimensional, noisy, and strongly nonconvex regimes.
S. Islam, Andreas Mang, Maxim A. Olshanskii· 1 citation