Jan 2026· arXiv.org· Vol abs/2601.16510· 4 citations· 213 references
Computer ScienceMathematics
TL;DR
This tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.
Abstract
Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost. This tutorial highlights a shift in optimization: using differentiable programming not only to execute algorithms but to learn how to design them. Modern frameworks such as PyTorch, TensorFlow, and JAX enable this paradigm through efficient automatic differentiation. Embedding first-order methods within these systems allows end-to-end training that improves convergence and solution quality. Guided by Fenchel-Rockafellar duality, the tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.
Optimization layers enable the incorporation of structured constraints and decision problems into learning systems. Training such systems requires differentiating through the embedded optimization problem, which can be challenging for general conic programs. We introduce dOPT, a solver-agnostic framework that, rather t...
Feng-Yu Yang, Connor W. Magoon, Tyler Watts et al.· 0 citations
Interior-point methods (IPMs) are among the most widely used algorithms for constrained optimization, yet their Newton-based search directions require costly second-order information and large linear-system solves. Learning to optimize offers cheaper updates learned from data, but the singular behavior of logarithmic b...
A. Madabhushi, Jia-Lin Liu, Min-Xin Zhang· 0 citations
This work reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization.
Yuan-Kun Chen, Zi-Fei Nie, Kang-Yu Lin et al.· 0 citations
A new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian that achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual, the first second-order method for this problem class attaining this rate.
Differentiating through parametric optimization problems is central to bilevel programming and meta-learning, often accomplished using approximate implicit differentiation. The implicit function theorem requires inverting a partial Jacobian of the optimality condition, which fails when there are many solutions. Nonethe...
Baptiste Plaquevent-Jourdain, Jalal Fadili, Antonio Silveti-Falls· 0 citations
This study presents a novel Deep Proximal Gradient Descent framework for ill-posed problems by employing a tailored second-order differentiable Input-Convex Neural Networks (ICNNs) as a learned regularizer, and introduces an innovative formulation that employs the learned residual to guide gradient descent.
Tian-Yi Ye, Guang-Yu Gao, Yang Li et al.· Inverse Problems· 0 citations
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
MIT News · Artificial Intelligence· news.mit.eduOct 13, 2026
Machine-learning networks can learn without detailed instructions from a designer. Those that do may offer insight into how the brain reorganizes itself during learning.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
