More Programs or More Rolls? Separating Coverage from Specialization in LLM Harnesses
Automated generation of LLM harnesses promises to improve inference through task specialization. Yet additional answer coverage can arise from repeated execution of the same program, making specialization difficult to identify. We introduce a controlled evaluation that separates answer coverage, repeatable task advanta...
Zi-Yang Xu, Haitian Zhong, Hao Zhou et al.
· 0 citations