Certifying Model Upgrades with Slice-Wise Non-Regression and Incumbent Fallback
An updated model can improve an aggregate metric while degrading a slice that matters to a downstream user. We study checkpoint selection subject to non-regression tolerances relative to a retained incumbent. The central distinction is between failing to detect harm and certifying non-inferiority: the former can releas...
Sheng-Wei Zhang, Tao Wu, Fei-Fei Qian
· 1 citation