We develop a tensor reduced-order modeling (TROM) framework for optimization-based inverse problems governed by parameter-dependent dynamical systems. The approach approximates the parameter-to-observation map directly in tensor-train format, using either TT-SVD or TT-Cross compression, and integrates the resulting representation into a regularized nonlinear least-squares formulation. Beyond accelerating forward evaluations, the low-rank tensor structure is used to reformulate the inverse problem in reduced coordinates, assemble the Gauss--Newton quantities without forming the full observation-space Jacobian, and perform TROM-based objective minimization over the discrete parameter grid. This tensor optimization step can be used either as a stand-alone approximate minimization procedure or as a data-informed initialization for a subsequent Gauss--Newton solve. The method is studied for two inverse problems: an inverse heat-transfer problem in a heterogeneous medium, where the unknown parameters describe the locations of multiple low-conductivity inclusions, and a FitzHugh--Nagumo parameter-estimation problem with a highly nonconvex optimization landscape. Numerical experiments assess the effects of ROM approximation error, measurement noise, regularization, initialization, spatial discretization, and increasing parameter dimension. The results show that TROM can reproduce the behavior of full-order inversion at a substantially reduced online cost. The experiments also demonstrate that reduced-coordinate inversion, tensor-based optimization, and appropriate regularization improve robustness in higher-dimensional, noisy, and strongly nonconvex regimes.
Tensor-train (TT) representations have emerged as an effective framework for mitigating the curse of dimensionality in the numerical solution of high-dimensional tensor differential equations. Among existing approaches, TT-cross methods are particularly attractive because they require only pointwise evaluations of the governing equations, naturally accommodate arbitrary nonlinearities, and avoid tangent-space projections and the numerical difficulties associated with nearly singular low-rank factors. However, existing TT-cross rank-truncation methods have been restricted to explicit time integration. Extending TT-cross methods to implicit schemes presents an obstacle: the collocation equations associated with the cross-selected fibers depend on neighboring fibers that are not part of the unknown set. Consequently, the resulting nonlinear system is not closed, preventing the direct application of standard implicit solvers. In this work, we introduce a principled fiber-dependency elimination framework that resolves this obstacle by expressing neighboring fibers as linear combinations of the cross-selected fibers through cross interpolation identities. The resulting formulation produces a closed collocation system while preserving the principal advantages of TT-cross methods. The proposed framework applies to both linear and nonlinear high-dimensional partial differential equations and is naturally combined with Newton iterations and rank adaptivity. Numerical experiments demonstrate rapid convergence of the dependency-elimination iterations, preservation of the temporal accuracy of implicit multistep schemes, and efficient implicit integration of high-dimensional nonlinear problems with full-order discretizations containing up to $10^{55}$ degrees of freedom.
This work develops tensor-train (TT) formulations for solving large-scale three-dimensional linear elasticity problems discretized by isogeometric analysis. By exploiting the tensor-product structure of the basis functions and the low-rank structure of geometry-dependent coefficient fields, the stiffness operator, mass operator, force vector, and displacement solution are represented in TT format. Two solution strategies are investigated: a block-operator formulation, in which the coupled elasticity operator is stored as separated TT blocks, and a single-operator formulation, in which the full coupled system is stored as one monolithic TT operator. A matrix-free three-field TT conjugate-gradient solver is introduced for the block formulation, while AMEn is used for the single-operator formulation. Numerical examples demonstrate substantial compression of both operators and solutions compared with conventional sparse full-grid representations, showing that TT-based formulations provide an efficient and scalable approach for large-scale three-dimensional elasticity simulations.
Q. Tran, Duc P. Truong, William W. Dai et al.· 0 citations
Nonlinear least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks. Such problems have a natural geometric interpretation, model predictions form a manifold in data space, while the chosen parameterization can introduce parameter-effects curvature that becomes a dominant source of nonlinearity. This exposes a limitation of the Levenberg-Marquardt (LM) method, its tangent-space step is applied as a straight update in parameter coordinates. Geodesic acceleration gives a second-order correction, but its removal of parameter-effect curvature is exact only in the infinitesimal-step limit. We propose a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) to improve this consistency for finite optimization steps. By reformulating the geodesic equation, RNC-LM extends geodesic acceleration to arbitrary-order corrections and constructs finite-step updates with progressively higher reparameterization consistency. A line search along the resulting RNC curve controls the traveled distance while keeping the cost close to standard LM. The method eliminates the tangential component of residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with the linear model prediction of LM. On classical nonlinear least-squares benchmarks, RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems. On a reaction-diffusion PINN failure-mode benchmark, it reduces the relative L2 error to the order of 1e-3 and recovers a physically meaningful solution. On a large-scale machine-learning potential-energy-surface fitting task, it achieves a 34-fold speedup over standard LM.
Gradient-based Bayesian inference methods require efficient access to Jacobian and adjoint-Jacobian operators of high-dimensional forward models. While multigrid solvers provide near-optimal complexity for elliptic partial differential equations, they are rarely available in forms compatible with automatic differentiation (AD). We develop a differentiable multigrid solver for steady-state diffusion-absorption problems and derive its adjoint operations analytically through the full multigrid hierarchy. The resulting solver, DMGS, is implemented in C++ and interfaced with JAX to provide efficient Jacobian-vector and vector-Jacobian products for variational inference in the NIFTy framework. We validate the approach on a 3D inverse problem involving diffuse radiative transfer in tissue, reconstructing an effective radiative source from Monte Carlo-simulated data. The reconstruction reproduces the data at a reduced chi-squared of 1.1 and generalizes to 32 independent validation datasets. Benchmarks against a JAX-native multigrid implementation show comparable runtimes and consistently lower peak memory for the hand-derived adjoint, with modest reverse-mode overhead. These results establish differentiable multigrid solvers as practical building blocks for variational inference in PDE-constrained problems.
Andrés Ramírez, Philipp Haim, David Föger et al.· 0 citations
We introduce a residual-driven procedure for training nonlinear-manifold reduced-order models for parametrized linear partial differential equations that, given prescribed latent and lifting spaces, identifies the nonlinear lifting without high-fidelity solution snapshots. The approximation is represented by a low-dimensional latent coordinate together with a nonlinear lifting into a richer reduced space. Rather than fitting the lifting to snapshot data, we determine it by minimizing a computable residual-based upper bound for the state error. For affinely parametrized operators, the resulting training objective admits an efficient offline--online decomposition, and the lifting update reduces to a sequence of low-dimensional weighted least-squares problems. Numerical evaluations on an advection--diffusion problem and a plane-strain elasticity benchmark show that the proposed approach substantially improves accuracy over linear subspaces. The resulting nonlinear models achieve accuracy comparable to snapshot-driven nonlinear-manifold training while avoiding high-fidelity snapshots in the lifting-identification stage.
Francesco A. B. Silva, J. Ragusa, T. Guo et al.· 0 citations
Differential equations arise in numerous applications, particularly within scientific and technical contexts. Systems of stiff, time-dependent ordinary differential equations constitute the focus of this work. Exponential integrators are designed to solve such equations by integrating the linear part exactly, while simultaneously approximating the nonlinear part through a linear combination of $\varphi$-functions. By utilizing an augmented stiffness matrix, state-of-the-art methods like KIOPS and RK2EXPINT solve the linear part and evaluate linear combinations of $\varphi$-functions for the nonlinear part in a single step, effectively reducing the computational effort to a single matrix exponential evaluation. However, these classical approaches assume that the system is represented by matrices and vectors, potentially not utilizing the underlying high-dimensional structure. Tensors address this limitation and offer significant storage efficiency through well-established decompositions like the Tensor Train (TT) format. This work provides a general framework for solving stiff, time-dependent systems directly within the TT format. Specifically, KIOPS-TT and RK2EXPINT-TT are developed as extensions of the original KIOPS and RK2EXPINT algorithms. This involves reformulating the scheme of explicit exponential Runge-Kutta integrators for tensors and augmenting the stiffness tensor to compute linear combinations of $\varphi$-functions acting on tensors through a single evaluation of the exponential function using Krylov subspace methods. Furthermore, it is shown that the underlying theory of the matrix methods remains valid, thereby enabling the transfer of key theorems to the tensor case. Numerical experiments confirm significant speed-ups for KIOPS-TT and RK2EXPINT-TT in low-rank scenarios compared to their classical counterparts.
R. Weigel, Tom-Christian Riemer, Martin Stoll· 0 citations