SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilities
SecProbe is introduced, a framework for adaptive evaluation that combines Item Response Theory (IRT) with on-demand synthesis of repository-scale vulnerability-repair tasks and achieves comparable agent ability estimates while requiring agents to solve up to 29.5\% fewer tasks.