Scaling Laws Do Not Predict Adversarial Robustness in LLM-Based Security Detection Systems

Abstract

Large language models (LLMs) are increasingly deployed as automateddefenders in security operations, generating detection rules from attacktelemetry. This paper originally claimed that model size does not predictadversarial detection robustness, based on benchmarking five open-weightmodels (3.8B-14B parameters) with DABS (Dual Adversarial Benchmark Score),a standardized 0-100 metric introduced in this work. Version 1.1 update (2026-09-07): A subsequent audit found that theoriginal five-model result was not reproducible under a corrected,seed-controlled protocol -- see the attached ERRATA.pdf for five specificcorrections, and a replication study. Re-running the original modelsunder seeding did not reproduce the original values (14-33% per-modeldeviation, model ranking scrambled), and expanding to a 13-model grid(1.2B-117B parameters) under the corrected protocol yields a numericallypositive but statistically non-significant trend (R²=0.28, p=0.058). Theoriginal claim is treated as not surviving replication. Full detail,code, and data: https://github.com/0xDanielSec/duel-framework DUEL (Dual Unified Evasion Loop) is an adversarial red-teaming frameworkin which an LLM Attacker generates synthetic Microsoft Sentinel-compatibletelemetry and an LLM Defender generates KQL detection rules across 38MITRE ATT&CK techniques. Framework, benchmark, and dataset available atthe repository above.