Jun 2026· arXiv.org· Vol abs/2606.31061· 0 citations· 18 references
MathematicsComputer Science
TL;DR
Online TT-ALS (Alternating Least Squares) is introduced, an algorithm that sequentially enforces orthogonality constraints that allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy and is suitable for low-latency real-time processing applications.
Abstract
Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data. Existing algorithms for computing TT decompositions can be categorized into two main types: conventional batch-based approaches and recursive online methods. In the context of streaming data, batch methods typically achieve higher reconstruction accuracy but often suffer from memory exhaustion, while online methods provide greater computational efficiency. In this work, we introduce Online TT-ALS (Alternating Least Squares), an algorithm that sequentially enforces orthogonality constraints. This approach allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy. Theoretically, we prove that enforcing these orthogonal gauge constraints guarantees monotonic decrease of the local objective function and temporal smoothness. Computationally, our deterministic single-sweep update reduces the rank dependence from quadratic to linear, achieving an overall complexity of $\mathcal{O}(I^{n-1} r)$. Experimental results demonstrate that the proposed method outperforms existing online techniques not only in terms of mathematical approximation accuracy but also in human perception-based video quality metrics. Furthermore, compared to recent deep learning-based paradigms, our algebraic approach achieves speedups of several orders of magnitude. Consequently, our method exhibits high computational efficiency and is suitable for low-latency real-time processing applications.
This work presents the tree tensor network Nyström (TTNN), an algorithm that extends recent research on streamable tensor approximation to the more general tree tensor network format, enabling a unified treatment of various existing methods.
Alberto Bucci, Gianfranco Verzella· Numerical Linear Algebra wit...· 3 citations
Tensor networks are powerful formats for compressing large-scale data. However, their application to general data processing has been limited by the difficulty of performing nonlinear operations. Here, we introduce iterative tensor network transformations (ITNTs), a general algorithmic framework for the element-wise evaluation of elementary and nonlinear filtering functions on data encoded as tensor trains (TTs), a class of tensor networks. Our approach operates entirely in the compressed domain, enabling efficient computation on exponentially large datasets while maintaining a controlled computational cost. We demonstrate its power in two key areas: (I) evaluating highly nonlinear elementary and filtering functions on a 3D reactive flow field, enabling high-fidelity reaction rate computation and region filtering, and (II) finding extrema in complex optimization problems, such as solving Max-SAT instances on spaces up to $2^{70}$ configurations. These results establish ITNT as a foundational tool that provides tensor network methods with the capability for general-purpose data science and large-scale optimization.
Xiao Wang, Tomohiro Hashizume, Pia Siegl et al.· 2 citations
A novel reformulation of the butterfly structure as a compact tensor network is proposed, leading to speedups of orders of magnitude and significant accuracy improvements for large matrices compared to state‐of‐the‐art low‐rank matrix and QTT tensor completion algorithms.
Navjot Singh, Edgar Solomonik, X. Li et al.· Numerical Linear Algebra wit...· 0 citations
Traditional model reduction techniques for neural network compression usually leverage existing matrix/tensor decomposition methods to explore the redundancy among model parameters, which may overlook the local structures of the models. In this work, we propose a novel low-rank approximation algorithm, called Dimension-Augmented Tensor-Train (DATT) decomposition, which integrates dimension slicing with the Tensor-Train decomposition. DATT allows different partitions to be represented using distinct, non-shared spanning sets. This enables the decomposition to adapt more flexibly to localized structure, potentially achieving better compression. With the localized low-rank structures, DATT provides a more compact representation of neural network weights. We evaluated the performance of DATT with the other state-of-the-art low-rank approximation-based model compression algorithms, and demonstrated that DATT outperforms them in terms of parameter reduction and model accuracy. In particular, for compressing ResNet-20 on CIFAR-10, DATT achieves a 77.00% reduction in parameters with only a 0.37% drop in top-1 accuracy.
Yu-Chuan Tai, Cheng-Yu Sie, Che-Rung Lee· Annual International Compute...· 0 citations
An efficient numerical approach for compressing a high-dimensional discrete distribution function into a non-negative tensor train (NTT) format and observing that the proposed NTT fitting procedure exhibits drastically faster convergence than an alternative multiplicative update method that has been previously proposed is observed.
Xun Tang, R. Dwaraknath, Lexing Ying· SIAM Journal on Scientific C...· 0 citations
A tensor ring alternating least squares with Tikhonov regularization subproblem (TRATR), a framework that incorporates Tikhonov regularization into the core-wise update of the TR-ALS, which demonstrates consistent improvements in relative standard error, peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM).