Evaluating MiniMax-M3 for Automated Web Penetration Testing on XBEN Benchmarks
Abstract
The use of automated penetration testing is growing as an important way to manage the rising expenses and technical requirements of today’s cybersecurity. The main issue addressed is the significant difference in the performance of Large Language Models (LLM) and the associated operational costs for performing autonomous security assessments. The goal of this study is to explore the effect of integrating the open-weight long-context model MiniMax-M3 into the PentestGPT framework. The research used a methodical experimental process with the XBEN benchmark suite, which consists of 104 web security challenges with different difficulty levels and vulnerability classifications. The execution followed a five-stage pipeline: benchmark loading, wiring the framework, planning with the LLM, executing the commands and analyzing the performance. Results indicate a total success rate of 71.2% with perfect scores on 5 vulnerability classes including SQL injection and JWT forgery. Cost analysis shows that this framework can greatly reduce the cost in comparison with the frontier-tier models. Overall, MiniMax-M3 provides an efficient and cost-effective option for automated vulnerability triage, instructional scenarios, and reproducible security evaluation harnesses under human-in-the-loop oversight.