Amortizing Scaling Law Construction Costs
A framework for efficient scaling law construction that formulates data collection as a Bayesian optimization problem is proposed, and metrics for comparing scaling law fitting methods under constrained compute budgets are introduced.