This work provides the first practical framework for learning lossless Fourier-domain filters at scale by reframing a hard combinatorial problem as a differentiable learning task by reformulating the hard combinatorial problem as a differentiable optimization task.
Abstract
Purpose. The Fourier transform-bounded Kolmogorov complexity proposed in the study by Terry-Jack [J. Software Eng. Appl. 15(10), 359 (2022)], as well as in that by Terry-Jack and O’Keefe [Physica D 453, 133824 (2023)], relies on finding the minimal subset of frequency coefficients that can exactly reconstruct the original image. However, we show that finding this minimal subset of coefficients is a combinatorial optimization problem that becomes computationally intractable for large images. As such, naive solutions such as greedy search are impractical at scale, limiting the feasibility and application of Fourier transform bounded Kolmogorov complexity. Methods. We reformulate the hard combinatorial problem as a differentiable optimization task via the introduction of smooth surrogates for non-differentiable components such as the quantization step and use a straight-through estimator (STE) to learn sparse binary masks (frequency filters) that allow for exact reconstruction of the original image via gradient-descent. Results. Experiments on elementary cellular automata, QR codes, a natural binary image, and random noise demonstrate that sparse Fourier masks capable of bit-exact reconstruction can be learned efficiently at scales that are impractical for a combinatorial or greedy search. Although the problem admits a linear LASSO reformulation that enables classical sparse-recovery methods, experiments against a FISTA baseline demonstrate the advantages of directly optimizing the original nonlinear objective via an STE. Conclusion. This work provides the first practical framework for learning lossless Fourier-domain filters at scale by reframing a hard combinatorial problem as a differentiable learning task.
Laplacian-regularized minimization is fundamental in signal processing and machine learning, but is limited by the dense and ill-conditioned nature of the graph Laplacian pseudoinverse. While the Laplacian itself is sparse, its pseudoinverse is dense and often ill-conditioned, rendering direct computation impractical at scale. Moreover, pseudoinverse learning is more challenging than Laplacian learning. To address this challenge, this paper considers the setting where the graph Laplacian is given and proposes a Difference-of-Convex Regularizer (DCR) graph learning framework that approximates the spectral action of the Laplacian pseudoinverse without direct inversion via regularized Maximum Likelihood Estimation (MLE). By reformulating Laplacian-Regularized Nonnegative Least Squares (LR-NNLS) through a dual representation, DCR decouples pseudoinverse learning from instance-specific inference and enables efficient primal solution reconstruction via a differentiable dual-guided learning scheme. We establish theoretical guarantees on stability and the existence of a unique fixed point for DCR algorithm. Numerical experiments demonstrate improved performance over convex solvers and graph filtering baselines and robust performance across diverse graph topologies.
A dimension-free version of the retained-energy form of the Mallat--Zeitouni conjecture is established, showing that the KL basis is within this factor of the optimal basis, and shows that the possible advantage of optimizing over all orthonormal bases vanishes as $d$ grows.
Minbo Gao, Zheng-Feng Ji, Cheng-Hua Liu· 0 citations
In this work, we consider the problem of sparse signal recovery known as compressed sensing using $\ell_1$-minimization.
We show how the $\ell_1$-minimization problem (also known as basis pursuit) can be transformed into an equivalent linear programming (LP) problem, and provide a proof of the equivalence of these two problems.
We conduct an experimental comparison of modern solvers (Gurobi, HiGHS, CPLEX, and Clarabel) for solving the LP problem on test data generated according to theoretical recovery guarantees for matrices with normally distributed elements. The results show that the open-source solver Clarabel is a competitive alternative to proprietary solvers in terms of speed.
We also propose a method for verifying the uniqueness of the obtained solution using an auxiliary quadratic programming problem with a strictly convex objective function.
A geometric interpretation of the uniqueness conditions is provided, and the application of the method is demonstrated on an example of a matrix with integer elements.
Anastasiia O. Storozhenko, P. Stetsyuk· KyivAcademUs2026· 0 citations
A universal, sample-optimal convergence theorem for the original BIHT algorithm is proved and a scalar lower bound is proved showing that any nontrivial corruption pattern, even one that involves only one flipped sign together with one clean sign, forces the iterates to oscillate indefinitely.
Abstract.
We study a sample complexity vs. conditioning tradeoff in modern signal recovery problems (including sparse recovery, low-rank matrix sensing, covariance estimation, and abstract phase retrieval), where convex optimization problems are built from sampled observations. We begin by introducing a set of condition numbers related to sharpness in the [Formula: see text] or Schatten-1 norm of nonsmooth formulations for these problems. Then, we show that these condition numbers become dimension- independent constants in each of the example signal recovery problems once the sample size exceeds some constant multiple of the recovery threshold. Structurally, this result ensures that the inaccuracy in the recovered signal due to both observation noise and optimization error is controlled. Algorithmically, such a result ensures that a new restarted mirror descent method achieves nearly dimension-independent linear convergence to the signal in terms of iterations. This new first-order method is general and applies to any sharp convex function in an [Formula: see text] or Schatten-[Formula: see text] norm for [Formula: see text].
Lijun Ding, Alex L. Wang· SIAM Journal on Optimization· 0 citations
BaKron is an efficient solver that combines anti-diagonal parallelism with a recursive divide-and-conquer construction that matches the cubic scaling of GPTQ while exploiting richer curvature information.