Can LLMs predict the outcomes of machine learning experiments?
This work evaluates how accurately large Language Models predict the impact of self-generated ML experiment ideas before these are put into training runs to show that current LLMs have useful but limited intuition about which ML ideas will work.