Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks
This work audits eight cybersecurity benchmarks across 10 proprietary, open-weight, and cybersecurity-specialized LLMs and shows that cybersecurity LLM benchmark scores are pipeline-dependent and motivate pipeline-aware auditing as a core requirement for reliable model evaluation.
Aymene Berriche, Cathrine Shalby, Mohannad J. Alhanahnah et al.
· 1 citation