The Price of Correlated Tests: How Strict Should a Model Release Gate Be?
This work treats the release gate as a design problem: choose how many tests a model must pass so that cleared models meet a stated reliability target, while keeping as many good models as possible, and gives a validation procedure that certifies a gate from labelled data even when the gate is chosen from a fixed short...