This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools, and presents a taxonomy-driven framework applicable to open-source and proprietary solutions.
Abstract
Rapid adoption of large language models (LLMs) in enterprise settings has introduced operational, security, and governance risks. As generative AI applications move from pilot to production, manual harm identification and mitigation are becoming difficult to scale. Although many tools support model evaluation, adversarial testing, runtime guardrails, and observability, the tooling landscape remains fragmented. Tools are typically designed for specific engineering tasks and described in technical terms that do not align with governance frameworks or risk taxonomies, making it difficult to determine which tools address which risks and where critical gaps remain. This paper proposes a structured protocol to automate AI risk mitigation through a taxonomy-driven analysis of open-source LLM evaluation and security tools. We map the capabilities of 21 prominent open-source tools to the 32 subcategories of the extended MIT AI Risk Mitigation and Response Taxonomy. An LLM-assisted retrieval-augmented generation pipeline analyzes source code and documentation to extract capabilities for each taxonomy category. Reliability assessment yielded moderate agreement (Fleiss'Kappa = 0.509) among three independent reviewers. The analysis reveals a highly skewed landscape in which tools cluster around technical and operational controls, while governance, legal and regulatory, and financial and market controls remain largely unaddressed. This motivates a layered risk-mitigation architecture combining tool-based controls with organizational and regulatory processes. The mapping protocol achieved an F1 score of 75.5% after majority voting. Overall, the study provides a practical mapping between enterprise AI risk categories and open-source mitigation capabilities, identifies where human oversight remains necessary, and presents a taxonomy-driven framework applicable to open-source and proprietary solutions.
Generative Artificial Intelligence systems—particularly those built on Large Language Models (LLMs)—have become central to modern enterprise computing, yet they carry with them a class of vulnerabilities that traditional cybersecurity models were never designed to address. Decoder-only transformer architectures process system instructions and untrusted user inputs as a single undifferentiated sequence of tokens, which makes them susceptible to direct and indirect prompt injections, jailbreaking attacks, and retrieval poisoning. Existing governance frameworks—including the NIST AI Risk Management Framework, the EU AI Act, and ISO/IEC 42001—offer valuable compliance guidance at the organizational level, but they stop short of providing the kind of execution-level security blueprints that engineering teams actually need.
This paper introduces the Prompt-Level AI Risk Governance Framework (PLAIRGF), a four-phase architectural model covering Prevention, Detection, Response, and Continuous Improvement. Rather than depending on simulated metrics to evaluate the framework, we validate PLAIRGF through mathematical formalization of key risk indicators—including Attack Success Rate (ASR), Risk Severity Score (R), the Governance Readiness Index (GRI), and a newly proposed Human-in-the-Loop Alignment Coefficient (η)—combined with rigorous defensive capability mapping and step-by-step operational trace walkthroughs of representative adversarial attack scenarios. The paper concludes with a structured alignment between PLAIRGF controls and international compliance standards, offering organizations a practical, audit-ready foundation for deploying LLM-based systems securely.
Keywords: Generative AI Security, Large Language Models, Prompt Injection, AI Governance, Retrieval-Augmented Generation, Human-in-the-Loop, PLAIRGF
Dr. Abdul Majid Farooqi Dr. Abdul Majid Farooqi, Ziya Anjum Ziya Anjum· International Scientific Jou...· 0 citations
Large Language Models (LLMs) are now deployed at an unprecedented scale across many critical sectors, rapidly transitioning from experimental AI tools to embedded components of production software systems. This accelerated adoption, often enabled by low-code integrations, has lowered technical barriers while simultaneously expanding the attack surface of modern applications, particularly when deployments occur without sufficient domain-specific security expertise. In many cases, security maturity has not progressed at the same pace as capability expansion, creating systemic exposure across confidentiality, integrity, and availability dimensions. To provide structured clarity amid this rapid growth, this paper presents a comparative and standards-aligned analysis of LLM security risks and defense mechanisms grounded in the OWASP GenAI Top-10 (2025). We systematically examine each vulnerability class, map representative attack patterns to primary mitigation strategies, evaluate their security property impact, and analyze practical limitations and implementation trade-offs. In addition, we introduce a severity-based assessment to prioritize risks according to operational and systemic impact, offering a quantitative perspective on defensive readiness. Our findings indicate that current mitigation strategies are predominantly reactive, concentrated at inference time, and unevenly distributed across the LLM lifecycle. Controls addressing training pipelines, supplychain dependencies, and autonomous system behaviors remain comparatively less mature and less standardized. By integrating vulnerability classification, defense mapping, severity prioritization, and trade-off analysis within a unified framework, this study provides actionable guidance for strengthening secure, resilient, and standards-driven LLM deployment in high-stakes environments.
Md Abdul Barek, Md Bajlur Rashid, A. K. I. Riad et al.· Annual International Compute...· 0 citations
The risk is not tied to any particular model but rather to the task category, and organizations should ask not which tool to trust, but instead whether LLM-generated automation code should be deployed without review.
Empirical evidence is provided that generative AI can effectively support security requirements engineering when embedded within human-centered workflows and organizational governance structures, offering practical insights for adoption in regulated software development contexts.
F. Martins, Elaine Venson· SIGSOFT FSE Companion· 0 citations
The increasing deployment of Large Language Models (LLMs) in critical infrastructure has introduced a class of security risks that remain insufficiently characterised and poorly tooled in practice. Attack vectors including prompt injection, jailbreaking, code execution facilitation, covert data exfiltration, and training data poisoning present quantifiable threats to systems that rely on LLM outputs, yet no lightweight, provider-agnostic tool exists to measure these risks systematically. This paper presents VectorSec, an open-source, web-based LLM security scanner built with Python and Dash. The tool executes a structured test suite of 320 adversarial prompts spanning 16 vulnerability categories aligned with the OWASP Top 10 for LLMs. Each response is scored through a four-stage pipeline combining pattern matching, sentiment analysis, semantic similarity, and secondary LLM verification. Evaluation across three open-source models demonstrates that pattern-matching-only filtering misses approximately 15% of High and Critical findings that semantic verification correctly identifies, motivating the multi-layer design. VectorSec provides interactive dashboard reporting, real-time progress tracking, and audit-ready PDF and CSV exports, lowering the barrier to structured LLM security assessment in operational settings.
M. Yamin· International Conference on...· 0 citations
A structured, research-backed blueprint for build-time security controls tailored specifically to AI-generated code is developed, organized across three interdependent layers: a pre-pull-request policy and governance layer; a build-time detection layer combining AI-augmented and traditional static analysis; and a prioritization and feedback layer applying application security posture management correlation to convert detection volume into developer-actionable risk reduction.
Bala Thripura Akasam· Journal of Cyber Security· 0 citations