F. Zarrinkalam

Beyond the Hype: Evaluating LLM Integration and Practical Limitations in Security Operation Centers

Large Language Models (LLMs) are increasingly being explored within Security Operation Centers (SOCs) to support text-heavy analytical work such as alert contextualization, incident summarization, and drafting investigative artifacts. Despite this interest, practitioners describe critical operational concerns, most notably hallucinations (plausible but incorrect outputs), opaque reasoning, and the verification effort required to safely use model-generated content in security workflows. In this paper, we present findings from semi-structured interviews with 20 SOC practitioners spanning frontline analysts, SOC managers, and tool developers. Participants report perceived time savings for low-stakes tasks that are quickly verifiable (e.g., summarizing logs or drafting initial investigative leads), but they consistently frame LLM outputs as preliminary drafts and suggestions rather than decision-grade conclusions. Participants also describe limited trust in LLMs for high-stakes security decisions due to unreliable outputs and unclear model reasoning, and they report relying primarily on ad-hoc verification norms and continuous human oversight rather than standardized mitigation procedures. Based on these interview-grounded accounts, we introduce a maturity rubric to characterize readiness for LLM integration and outline a research agenda emphasizing auditability and transparent explanation mechanisms to support safer adoption in SOC workflows.

APT-CGSV: APT Detection via Contrastive GNNs and Semantic Validation

Advanced Persistent Threats (APTs) increasingly employ “Living-off-the-Land” techniques, rendering traditional signature-based detection ineffective. While query-based threat hunting using system provenance graphs provides essential behavioral context, existing approaches, specifically Graph Neural Network (GNN) based methods, can suffer from a critical vulnerability to structural mimicry, where benign administrative tasks topologically resemble malicious campaigns. Large Language Models (LLMs) offer the deep semantic reasoning required to resolve this ambiguity, but applying them to raw enterprise logs is constrained by prohibitive latency, limited context, and data privacy constraints. In this paper, we propose a novel neuro-symbolic framework that bridges this gap via a highly efficient two-stage cascade architecture. First, a self-supervised contrastive Graph Isomorphism Network with Edge features (GINE) acts as a high-recall structural filter. Trained on a curated dataset of just 24,000 subgraphs, which is much smaller than those used by existing supervised learning-based approaches, our encoder learns robust representations resilient to graph noise and successfully eliminates 87% to over 99% of the benign structural mimics in our curated test sets. Second, the reduced candidate set is processed by a localized LLM utilizing deterministic artifact matching and constrained Chain-of-Thought (CoT) behavioral matching for semantic reasoning. Evaluated across diverse operating systems (Windows, Linux, FreeBSD) using our curated test sets from the DARPA TC3 and OpTC datasets, our framework achieves superior precision and F1-scores as compared to existing baseline systems. By combining the data efficiency of contrastive GNNs with the semantic depth of localized LLMs, we provide a scalable, highly accurate, and efficient threat-hunting solution.

Diagnosing and Repairing Factual Errors in RAG under Budget Constraints

Retrieval-Augmented Generation (RAG) improves the factuality of large language models by grounding responses in external evidence, yet real-world deployments remain fragile. Failures often stem from missing or weakly relevant evidence, as well as from generation that does not faithfully reflect the retrieved context. Many existing approaches rely on fine-tuning, privileged access to internal model signals, or resource-insensitive escalation strategies, which limits their practicality in black-box and budget-constrained settings. We propose D2R-RAG (Diagnose-to-Repair RAG), a model-agnostic and resource-aware framework that combines lightweight failure diagnosis with adaptive repair. D2R-RAG derives interpretable failure signatures from observable signals in the query, retrieved evidence, and generated response, and then selects from a small set of corrective actions under explicit latency and VRAM constraints. Experiments on FEVER and HotpotQA show that D2R-RAG improves reliability over recent baselines and achieves better accuracy--efficiency trade-offs across multiple compute budgets. The code is available at https://github.com/CyberScienceLab/D2R-RAG/.

Diffusion-based generative modeling for expert team formation