Jun 2026· International Joint Conference on Computer Science and Software Engineering· pp. 85-90· 0 citations· 18 references
Abstract
Penetration testing is still highly dependent on trained professionals (who are required to manually coordinate heterogeneous security tools, multi-phase findings, and actionable reports) that is both time-consuming, expertise-oriented and inadequate to the speed of current threats. The framework described in this paper, SHARKAPT, is an autonomous AI-based penetration testing framework that uses the Model Context Protocol and large language model orchestration to conduct end-to-end automated security testing of both network and web application-related systems. SHARKAPT uses a multi-agent architecture based on LangGraph with five specialised agents: reconnaissance, network penetration testing, web application testing, exploitation as well as report generation and a decision engine that dynamically chooses 60+ real-world security tools. Google Gemini 3.1 Pro evaluated empirically against Metasploitable2 and OWASP Juice Shop has coverage of 71.7% of the Open Web Application Security Project Top 10 2021 challenges (9/10 categories), 79.1% Metasploitable2 known-vulnerability coverage, a 99.1% step success rate on the web session (111 steps), and a 94.5% step success rate on the network session (73 steps). All finding results are rated using a deterministic CVSS v3.0 calculator and stored in organised HTML reports containing per-finding evidence.
MTGuard is proposed, a hybrid analysis-based defense framework designed to safeguard the use of MCP tools in LLM agents by leveraging lifecycle-aware static-dynamic co-analysis and effectively mitigates multiple categories of harmful tool use across different LLM agents while maintaining performance on benign user tasks.
Ping He, Yuexiang Xie, Yaliang Li et al.· 0 citations
Large language model agents driving security tool suites over the Model Context Protocol are increasingly common. Yet the factors that bound their capability remain poorly characterized: how much depends on the model versus the client that drives it, whether constraining the agent to the orchestrator's own tools helps, and where capability is limited by reasoning rather than by missing tools. Using HexStrikeAI, an open-source orchestrator that exposes 150+ tools, as a testbed, we follow a methodology that evaluates the system, diagnoses its failures, and applies targeted improvements. We run 86 picoCTF challenges across seven categories and three difficulty tiers, under three tool-access regimes and three model/client configurations (774 trials). We then apply corrections to existing tools, agent-behavior changes, and eleven new capability tools, and re-run the previously-unsuccessful trials. The diagnosis isolates the driving client as a first-order factor for a fixed model (a 2.1 * gap between two DeepSeek clients) and a monotonic difficulty gradient, with the largest gains in the mid tier. The overall solve rate rises from 55.4% to 72.0%, and every configuration improves significantly (paired McNemar p<0.001, non-overlapping 95% confidence intervals). The residual failures are reasoning- or environment-bound rather than missing-tool. A 60-run stability sub-study finds single-run verdicts reproducible (17/20 unanimous). We discuss what the results imply for how such orchestrators should be evaluated, and we are explicit about the limits: the study uses a single benchmark, the fixes were tuned on the same challenges they were evaluated on, and the client effect is demonstrated for one model only, so its generality to other models remains a hypothesis.
Romain Gerard, Assmaa Zeghaider, Yan Guo· 0 citations
Testing RESTful API is increasingly complicated but indispensable to quality assurance of cloud-native applications. This paper reports a multi-agent system called MASTEST that combines LLM-based intelligent agents and programmed agents to automate REST API testing. They form a complete tool chain covering the whole workflow of REST API test with API specification in the OpenAPI Swagger format as the input. It also incorporates human testers in the process to review and correct LLM generated test artefacts to control the quality of testing activities. MASTEST is evaluated on two LLMs, GPT-4o and DeepSeek V3.1 Reasoner with five public APIs. Its performances on various testing activities are measured by a wide range of metrics, including adequacy and coverage metrics, the syntax and data type correctness of generated test scripts, the usability of LLM generated test cases and scripts, as well as the bug detection ability. Experiment results demonstrated that both DeepSeek and GPT-4o achieved a high overall performance but had strengths and weaknesses on different testing activities. MASTEST generated test cases achieved 94% and 98% unit test coverage and 79% and 78% system test coverage for GPT-4o and DeepSeek respectively in comparison with human designed test cases. The generated test scripts maintained 100% syntax correctness and only required minimal manual edits for semantic correctness. The generated test scripts contain assertions on the expected status code as well as contents in the response messages. They are highly capable of detecting bugs in the REST APIs. Experiment data shows that the bug detection rates are between 2.13 to 4.50 per operation. These findings indicate that MASTEST is highly efficient and effective.
Xiaoke Han, Hong Zhu· International Conference on...· 1 citation· ⚡1
This study investigates whether a frontier LLM can generate Dockerfiles and Docker Compose configurations for multi-service applications using repository contents without access to developer-authored deployment artifacts and analytically derives a minimal explicit deployment specification for information that cannot be reliably inferred from repository artifacts.
Oleg Grynets, Kyrylo Fursov, V. Lyashkevych et al.· 0 citations
: Integrating Large Language Models (LLMs) into production-grade business systems requires a clear boundary between deterministic application logic and non-deterministic AI behavior. This paper introduces a runtime architecture that extends the BAB (Business Application Builder) framework on Jakarta EE to support agent-augmented applications hosted alongside ontological-driven generated cores. The core of the solution, the AgentRuntimeManager , moves beyond simple API wrappers by providing a robust environment for agent lifecycles, asynchronous event dispatching, and error handling through configurable retry policies. To ensure system stability, we decouple LLM providers (such as OpenAI and Ollama) using an abstraction layer built on LangChain4j, allowing for runtime provider switching without redeploying the core logic. We demonstrate the practical utility of this architecture through a self-healing log-monitoring chain. In this setup, agents intercept high-severity log events and, through a structured analysis service, propose corrective actions. A specialized SqlErrorHealingAgent illustrates how LLM-proposed fixes can be filtered through layered safety constraints, such as the automatic blocking of destructive SQL operations like DROP or TRUNCATE, before any action is committed. The result is an architectural pattern that lets model-driven rigor and LLM agent flexibility coexist in enterprise environments without compromising either.
Borivoj Bogdanović, M. Segedinac, Z. Konjović et al.· SINTEZA· 0 citations
VulnScanner Pro is a self-hosted assessment tool that tries to sit in the gap between the two extremes of manual penetration testing and automated security assessment, and recommends four recommendations for fixing the six missing HTTP security headers.
Athili Laxmi Lavanya, Mycharla Madhavkumar, D. Gangadhar· International Journal for Re...· 0 citations