When Are Predictions Enough? An Evaluation Protocol for Frozen Expert Composition
Practitioners often combine independently trained models while keeping their weights frozen. These experts are commonly combined through their final predictions, although their internal features may retain useful information that those predictions omit. We ask when combining features improves on combining predictions,...