Guan-Hua Zhang

Predictable performance is suboptimal, and what it means for model routing

This work takes a model's self-confidence, the probability it assigns to its chosen answer, as a baseline and study how much a performance predictor can improve on it, and shows that any performance predictor that outperforms a model's self-confidence can be used to improve the model's loss.