This work contributes an empirical, practitioner-driven analysis of LLM use across SOC roles and organizations and derives concrete design and integration requirements for human-centered, operationally safe LLM-assisted security operations.
Abstract
Security Operations Centers (SOCs) process large volumes of security events, requiring analysts to accurately detect and assess ongoing cyberattacks under time pressure. Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations, yet their practical suitability for real-world SOC workflows remains poorly understood. To address this gap, we conducted 25 semi-structured interviews with SOC practitioners who had prior experience with LLMs, complemented by interactive scenarios to anticipate challenges and identify opportunities for the responsible integration of LLM-based tools into SOC workflows. We identified 15 LLM use cases grouped into six functional categories. While LLMs are valued for automating repetitive, low-level tasks such as report automation, practitioners rate high-impact tasks such as incident analysis as not yet feasible, reporting limitations in technical depth, context awareness, and organization-specific knowledge. They locate these limitations less in the models than in the readiness of their SOCs and human factors driving over-reliance. Despite concerns, practitioners express a strong willingness to adopt LLMs, describing competitive pressure that leaves few alternatives. This work contributes an empirical, practitioner-driven analysis of LLM use across SOC roles and organizations and derives concrete design and integration requirements for human-centered, operationally safe LLM-assisted security operations.
Large Language Models (LLMs) are increasingly being explored within Security Operation Centers (SOCs) to support text-heavy analytical work such as alert contextualization, incident summarization, and drafting investigative artifacts. Despite this interest, practitioners describe critical operational concerns, most notably hallucinations (plausible but incorrect outputs), opaque reasoning, and the verification effort required to safely use model-generated content in security workflows. In this paper, we present findings from semi-structured interviews with 20 SOC practitioners spanning frontline analysts, SOC managers, and tool developers. Participants report perceived time savings for low-stakes tasks that are quickly verifiable (e.g., summarizing logs or drafting initial investigative leads), but they consistently frame LLM outputs as preliminary drafts and suggestions rather than decision-grade conclusions. Participants also describe limited trust in LLMs for high-stakes security decisions due to unreliable outputs and unclear model reasoning, and they report relying primarily on ad-hoc verification norms and continuous human oversight rather than standardized mitigation procedures. Based on these interview-grounded accounts, we introduce a maturity rubric to characterize readiness for LLM integration and outline a research agenda emphasizing auditability and transparent explanation mechanisms to support safer adoption in SOC workflows.
Elnaz Rabieinejad, Ali Dehghantanha, F. Zarrinkalam et al.· 0 citations
This work presents the first results from an exploratory assessment of GenAI-aware threat modeling methods in a Small and Medium Enterprise (SME) setting and reports practitioners' perceptions of the usability and integration of these methods in SME development workflows, including their perceived effort and adoption challenges.
Nicolás E. Díaz Ferreyra, M. Kumar, Nohemí Villarreal et al.· 0 citations
An evaluation of five state-of-the-art LLMs on OmniVul reveals distinct performance gaps, with top-1 accuracy remaining below 50% on average for vulnerable code detection and CVE identification, demonstrating that current models lack critical reasoning capabilities for reliable vulnerability assessment.
Vishnu Teja Kandalam, Viet Duong, Xiaochang Li et al.· Proceedings of the 32nd ACM...· 0 citations
Large Language Models (LLMs) are now deployed at an unprecedented scale across many critical sectors, rapidly transitioning from experimental AI tools to embedded components of production software systems. This accelerated adoption, often enabled by low-code integrations, has lowered technical barriers while simultaneously expanding the attack surface of modern applications, particularly when deployments occur without sufficient domain-specific security expertise. In many cases, security maturity has not progressed at the same pace as capability expansion, creating systemic exposure across confidentiality, integrity, and availability dimensions. To provide structured clarity amid this rapid growth, this paper presents a comparative and standards-aligned analysis of LLM security risks and defense mechanisms grounded in the OWASP GenAI Top-10 (2025). We systematically examine each vulnerability class, map representative attack patterns to primary mitigation strategies, evaluate their security property impact, and analyze practical limitations and implementation trade-offs. In addition, we introduce a severity-based assessment to prioritize risks according to operational and systemic impact, offering a quantitative perspective on defensive readiness. Our findings indicate that current mitigation strategies are predominantly reactive, concentrated at inference time, and unevenly distributed across the LLM lifecycle. Controls addressing training pipelines, supplychain dependencies, and autonomous system behaviors remain comparatively less mature and less standardized. By integrating vulnerability classification, defense mapping, severity prioritization, and trade-off analysis within a unified framework, this study provides actionable guidance for strengthening secure, resilient, and standards-driven LLM deployment in high-stakes environments.
Md Abdul Barek, Md Bajlur Rashid, A. K. I. Riad et al.· Annual International Compute...· 0 citations
: In every SOC team, a shift starts with hundreds, sometimes thousands, of alerts, and with even more during cyber incidents. In traditional SOC environments, most of these alerts need manual analysis and verification before action is taken. We live in the age of automation and AI, which SOC teams and attackers alike can use. When an attack is conducted with AI tools and automation, this can lead to a shorter response time [1]. Nowadays, there is a lack of IT personnel everywhere, especially those with adequate knowledge. SOC teams face the same personnel issue. The situation is even worse in 24/7 environments [2]. Night shifts are the hardest: false positives come up, people are tired because they normally sleep at these hours, and real threats sometimes go unnoticed. Traditional tools, such as SIEM, IPS/IDS, and EDR, are still part of every SOC and continue to detect threats, but cyber analysts must still perform most response steps by hand [3]. To overcome the problem where analysts have to work manually, SOAR platforms come into play. SOAR platforms connect to traditional SOC tools and leverage their data to automate repetitive tasks (such as IP reputation checks, WHOIS lookups, and hash verifications). In addition, SOAR introduces structure into incident response processes. In this way, security analysts have more time to do other work. This paper will examine the advantages and consequences of adopting SOAR in SOC, including process efficiency, process standardization, and expand-ability. This paper also discusses automation risks that are often not covered in vendor manuals [4]. Using SOAR can greatly reduce response time, sometimes by nearly half. Initial deployment typically takes several months (typically 3-6). The most important benefit for analysts is that they lose less time on repetitive tasks. The size of this benefit depends mostly on how well traditional SOC tools were prepared for integration with the SOAR platform and how well the integration went.
Boban Stojanović, Marko Premović, Igor Petrović· SINTEZA· 0 citations
The growing complexity and frequency of cyberattacks make cybersecurity risk assessment an increasingly demanding task for organisations, requiring substantial expertise, resources, and adherence to established standards. This work explores the applicability of Large Language Model (LLM) to cybersecurity risk assessment, with a focus on threat identification and risk scoring. The paper presents a standalone consistency analysis across five models, measuring accuracy and stability under lexical, structural, and noisy prompt perturbations using an OWASP-oriented rubric. Building on the analysis results, we present a modular LLM-based system that combines Retrieval-Augmented Generation, MITRE ATT&CK-Aligned threat evaluation, rubric-constrained risk scoring, and a Judge Reviewer, orchestrated through a Beliefs–Desires–Intentions control loop. The validation against incidents from the VERIS and EuRepoC datasets highlights limitations and weaknesses, and allows identifying the architectural and structural mitigations that can reduce prompt sensitivity in LLM-based risk assessment.