Quantifying Depth Sufficiency in Residual Neural Networks: A First-Order Criterion
This work defines first-order residual depth saturation as the absence of a strict local decrease from every admissible insertion, and proves residual non-degeneracy is necessary and sufficient: additional depth has first-order value exactly when conditional activation gradients have a nonzero projection onto at least...