Results indicate that LLMs provide a practical approach for automated vulnerability identification in JavaScript code, particularly when combined with task-aligned supervision, though they should complement rather than replace existing security analysis workflows due to limited recall and uneven performance across vulnerability types.
Abstract
JavaScript powers approximately 98.8% of all websites, making vulnerabilities in its code a significant security risk, yet existing detection approaches such as Static Application Security Testing (SAST) tools often fail to identify many real-world vulnerabilities when applied to isolated code snippets. This paper presents an empirical study of Large Language Model (LLM)-based vulnerability identification for JavaScript programs, evaluating three LLM families (Gemini 1.5 Flash, GPT-4o Mini, DeepSeek-R1-Distill-Llama-8B) across multiple prompting strategies (zero-shot, chain-of-thought, few-shot) and fine-tuning approaches on a dataset of 1,125 JavaScript code snippets spanning five Common Weakness Enumeration (CWE) categories: Injection (CWE-74), OS Command Injection (CWE-78), Cross-Site Scripting (CWE-79), SQL Injection (CWE-89), and Uncontrolled Resource Consumption (CWE-400). Our experiments show that LLMs substantially outperform traditional SAST tools on snippet-level vulnerability identification, with a fine-tuned Gemini 1.5 Flash model achieving 60% detection accuracy compared to near-zero performance from rule-based analyzers. We find that fine-tuning improves accuracy from 29% to 60%, Chain-of-Thought prompting benefits reasoning-capable models such as GPT-4o Mini, few-shot prompting is effective for polymorphic vulnerabilities such as Cross-Site Scripting, and performance varies across vulnerability categories, reaching up to 84% accuracy for structured vulnerabilities such as SQL Injection. These results indicate that LLMs provide a practical approach for automated vulnerability identification in JavaScript code, particularly when combined with task-aligned supervision, though they should complement rather than replace existing security analysis workflows due to limited recall and uneven performance across vulnerability types.
A security assessment on 75 FastAPI backends generated by three contemporary LLMs revealed a disconnect between functional correctness and secure logic, which is interpreted as a review-risk pattern, which is called the human-in-the-loop paradox.
Abdul Ali Khan, S. Rauti, T. Mäkilä· IEEE Access· 0 citations
Experimental results show that AST-based structural features substantially improve recall compared with the TF-IDF baseline, while the combined TF-IDF and AST representation maintains this improved performance.
Vani Pasupula, M. N. V. Manikanth, Nagaraju Vassey· International Journal of Cre...· 0 citations
Investigating Behavior Trees (BTs) as an alternative intermediate representation for LLM-based vulnerability detection suggests that BTs can provide a compact and useful structured representation for vulnerability detection with quantized, locally deployable LLMs.
Enna Bašić, A. Giaretta· ACM Workshop on Artificial I...· 0 citations
Large language models (LLMs) demonstrate strong capabilities in code-related tasks, however their effectiveness in software vulnerability detection (SVD) remains poorly understood due to inadequate evaluation frameworks. Existing benchmarks suffer from training data contamination, isolated function evaluation without c...
Arastoo Zibaeirad, Rodrigo Pato Nogueira, Marco Vieira· IEEE Transactions on Reliabi...· 0 citations
SETYPE is presented, a semantics-aware type system that can be derived directly from source code based solely on the meanings of symbols and expressions in natural language that achieves 87% detection precision and 88% detection accuracy on real-world applications.
An on premise Artificial Intelligence (AI)-based model for vulnerability detection in source code, designed to ensure there is efficiency in identifying potential weaknesses, and deployed locally within a Dockerized environment.
D. Sako· International Journal of Com...· 0 citations