The critical failure modes in deployed large language models (LLMs) are cross-dimensional: a model can score 99.3 in safety alignment while refusing one in three benign queries, or improve across every capability metric while losing 21 points in privacy. Existing evaluation frameworks that assess safety, security, and...
Fatih Deniz, Yazan Boshmaf, Dorde Popovic et al.· 0 citations
SSP-Bench reveals systematic failures of static evaluation, including near-zero correlation in safety rankings due to construct mixing, strong safety--over-refusal coupling, and hidden within-family regressions, and shows that static benchmarks can misrepresent model behavior, motivating dynamic, deployment-relevant ev...
Fatih Deniz, Yazan Boshmaf, Issa M. Khalil· 0 citations
This work audits eight cybersecurity benchmarks across 10 proprietary, open-weight, and cybersecurity-specialized LLMs and shows that cybersecurity LLM benchmark scores are pipeline-dependent and motivate pipeline-aware auditing as a core requirement for reliable model evaluation.
Aymene Berriche, Cathrine Shalby, Mohannad J. Alhanahnah et al.· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.