Does Random Search Discover Task Experts? Revisiting Neural Thickets
The results suggest that random weight perturbations may be ineffective at uncovering task experts around pretrained models, and that task adaptation requires more sophistication than random search.