Skip to content

lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers

Sep 2026 · arXiv.org · Vol abs/2609.33630 · 0 citations · 27 references
Computer Science Mathematics Engineering

TL;DR

This work reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization.

Abstract

Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.

View source

Similar papers

#machine learning Preprint Sep 2026

dOPT: Differentiating Conic Optimization via Geometric Reduction

Optimization layers enable the incorporation of structured constraints and decision problems into learning systems. Training such systems requires differentiating through the embedded optimization problem, which can be challenging for general conic programs. We introduce dOPT, a solver-agnostic framework that, rather t...

Feng-Yu Yang, Connor W. Magoon, Tyler Watts et al. · 0 citations
Preprint Aug 2026

Envelopt: Constrained Convex Composite Optimization

We introduce Envelopt, a globally convergent iterative framework for a broad class of structured optimization problems where a smooth objective is augmented by a nonsmooth convex regularizer composed with a smooth mapping, and the variables are subject to general smooth constraints. All smooth functions may be nonconve...

Alberto De Marchi, Dominique Orban · 0 citations
Preprint Sep 2026

Primal-Dual Inexact Newton-MR for Nonconvex Optimization with Equality Constraints

This work proposes an inexact primal-dual Newton method with an inner solver based on the conjugate residual (CR) method, which handles indefiniteness in the subproblem directly as it arises and establishes worst-case global convergence guarantees and strong empirical performance on large-scale, nonconvex problems.

Oscar Smee, Fred Roosta · 0 citations
Preprint Sep 2026

PDNQP: A GPU-based Factorization-free Method for Large-scale Nonconvex Quadratic Programming

PDNQP is presented, a factorization-free first-order solver for finding stationary points of large-scale nonconvex quadratic programs and its key ingredients are a residual-form reformulation that avoids penalty-dependent normal matrices and preserves a fixed sparse constraint operator across outer iterations, and an a...

Zi-Xi Chen, Hai-Hao Lu · 0 citations
#machine learning Preprint Sep 2026

Learned Preconditioning for a Primal-Dual Interior-Point Method

Interior-point methods (IPMs) are among the most widely used algorithms for constrained optimization, yet their Newton-based search directions require costly second-order information and large linear-system solves. Learning to optimize offers cheaper updates learned from data, but the singular behavior of logarithmic b...

A. Madabhushi, Jia-Lin Liu, Min-Xin Zhang · 0 citations
Aug 2026

Stochastic ADMM with Balanced Augmented Lagrangian Method for Nonconvex and Nonsmooth Finite-Sum Optimization

In this paper, we propose a balanced augmented Lagrangian method based on accelerated stochastic ADMM (b-ASADMM) to efficiently solve structured separable nonconvex optimization problems subject to linear constraints. The objective function in this problem comprises potentially nonsmooth and smooth functions, where the...

Qiaoling Zhang, Chuang Yang, Hu Shao · 0 citations

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.