Stochastic Nonconvex Bilevel Optimization: Improved Rates Without Rare-Visit Assumption
This work shows that a simple denominator-only regularization of the DBGD multiplier eliminates the need for such an assumption while preserving fast convergence rates, and improves upon the best assumption-free complexities.