Skip to content

Category

software testing

633 papers

#computer vision Review Jul 2026

Vibe Coding in Software Development: A Multivocal Literature Review

Vibe coding is a software development practice in which developers state intent in natural language and large language models generate code. It is often framed as one-shot prompting, but the evidence describes an intent-driven, iterative workflow whose outcomes depend on how generated code is evaluated and governed. Knowledge of how vibe coding is defined, practiced, and governed is scattered across academic and practitioner sources, and, to our knowledge, existing reviews have not yet integrated both evidence streams. We conducted a multivocal literature review of peer-reviewed and grey literature following established guidelines. Searches spanned 2022 to October 2025. After screening, credibility assessment, and snowballing, 47 sources were retained (28 peer-reviewed and 19 grey) and analyzed through descriptive mapping and thematic synthesis across eight research questions. Vibe coding is consistently described as an iterative generation-evaluation-revision loop rather than a one-shot activity, and developer work shifts from writing code towards specification, supervision, and validation. Short-term productivity and time-to-prototype gains are reported in 21 of 47 sources (45%), while evidence on maintainability, long-term quality, and safeguard effectiveness remains limited. Evidence is strongest for prototyping and user-interface work and weakest for production, data-intensive, and safety-critical use, and tool visibility does not imply effectiveness. This is one of the first reviews to integrate peer-reviewed and grey literature on vibe coding under a single documented protocol. Future work should evaluate safeguard effectiveness, study session-level dynamics and long-term maintainability, and test vibe coding in production, data-intensive, and safety-critical settings.

Shahbaz Siddeeq, Muhammad Waseem, Kai-Kristian Kemell et al. · 0 citations

Context Before Code: An Experience Report on Vibe Coding in Practice

Code-generating tools are increasingly used in software development, yet experience reports on conversational"vibe coding"under production constraints remain limited. This paper presents an experience report from a small full-stack team that applied contextual prompting and explicit architectural constraints to build (i) a multi-project agent learning platform designed for sustained, production-oriented use and (ii) an academic retrieval-augmented generation system. The agent platform supports multiple isolated projects, each with structured memory and background processing, thereby enforcing project-level isolation. The RAG system provides citation-grounded answers, role-based access control, and evaluation tracking. Across both systems, vibe coding accelerated scaffolding and integration. However, the generated code often under-specified isolation rules and infrastructure constraints when these were not explicitly defined. Consequently, aspects such as multi-tenancy, access control, memory policies, and asynchronous processing required deliberate architectural design and verification. We observe a shift in engineering effort from boilerplate implementation toward constraint specification and enforcement auditing. We also identify recurring architectural"non-delegation zones"where conversational code generation remains insufficient for production reliability.

Md Nasir Uddin Shuvo, M. Islam, Mahade Hasan et al. · 0 citations
#computer vision Open access Mar 2024

LLM-based agents for automating the enhancement of user story quality: An early report

In agile software development, maintaining high-quality user stories is crucial, but also challenging. This study explores the use of large language models to automatically improve the user story quality in Austrian Post Group IT agile teams. We developed a reference model for an Autonomous LLM-based Agent System and implemented it at the company. The quality of user stories in the study and the effectiveness of these agents for user story quality improvement was assessed by 11 participants across six agile teams. Our findings demonstrate the potential of LLMs in improving user story quality, contributing to the research on AI role in agile development, and providing a practical example of the transformative impact of AI in an industry setting.

Zheying Zhang, M. Rayhan, Tomas Herda et al. · 48 citations · ⚡4
#artificial intelligence Open access Jun 2025

AI and Agile Software Development: From Frustration to Success - XP2025 Workshop Summary

The full-day workshop on AI and Agile at XP 2025 convened a diverse group of researchers and industry practitioners to address the practical challenges and opportunities of integrating Artificial Intelligence into Agile software development. Through interactive sessions, participants identified shared frustrations related to integrating AI into Agile Software Development practices, including challenges with tooling, governance, data quality, and critical skill gaps. These challenges were systematically prioritized and analyzed to uncover root causes. The workshop culminated in the collaborative development of a research roadmap that pinpoints actionable directions for future work, including both immediate solutions and ambitious long-term goals. The key outcome is a structured agenda designed to foster joint industry-academic efforts to move from identified frustrations to successful implementation.

Tomas Herda, Victoria Pichler, Zheying Zhang et al. · 3 citations
#computer vision Open access Jun 2025

LLM-based Multi-Agent System for Intelligent Refactoring of Haskell Code

Refactoring is a constant activity in software development and maintenance. Scale and maintain software systems are based on code refactoring. However, this process is still labor intensive, as it requires programmers to analyze the codebases in detail to avoid introducing new defects. In this research, we put forward a large language model (LLM)-based multi-agent system to automate the refactoring process on Haskell code. The objective of this research is to evaluate the effect of LLM-based agents in performing structured and semantically accurate refactoring on Haskell code. Our proposed multi-agent system based on specialized agents with distinct roles, including code analysis, refactoring execution, verification, and debugging. To test the effectiveness and practical applicability of the multi-agent system, we conducted evaluations using different open-source Haskell codebases. The results of the experiments carried out showed that the proposed LLM-based multi-agent system could average 11.03% decreased complexity in code, an improvement of 22.46% in overall code quality, and increase performance efficiency by an average of 13.27%. Furthermore, memory allocation was optimized by up to 14.57%. These results highlight the ability of LLM-based multi-agent in managing refactoring tasks targeted toward functional programming paradigms. Our findings hint that LLM-based multi-agent systems integration into the refactoring of functional programming languages can enhance maintainability and support automated development workflows.

Shahbaz Siddeeq, Muhammad Waseem, Z. Rasheed et al. · 4 citations
#computer vision Book Open access Apr 2026

From Business Meetings to Requirement Artifacts: An Agentic AI Approach with MARARE

This paper presents MARARE, a real-time multi-agent system that transforms meeting dialogues into structured software requirements. One agent interacts with participants, while background agents extract and verify requirements collaboratively. Evaluation using the LLM-as-a-Judge method across five meetings (5–8 minutes each) shows a mean coverage of 80.0 ± 11.2 % (mean ± SD), semantic similarity of 0.86 ± 0.05, and hallucination rate of 14.3 ± 6.2 %. Preliminary results indicate performance differences across LLMs, suggesting that model choice influences coverage, consistency, and hallucination rates.

Malik Abdul Sami, Gessé Evangelista, Kai-Kristian Kemell et al. · 0 citations
#computer vision Review Aug 2026

REFINE: A Multi-Agent LLM Approach for Evidence-Guided Code Refactoring

Large Language Models (LLMs) offer new opportunities for automated code refactoring. However, generated changes must reduce targeted quality problems without introducing new issues or altering behaviour-relevant code structures. We introduce REFINE (Refactoring with Evidence-aware Flow for Integrated ageNtic Execution), a tool-agnostic, evidence-aware multi-agent approach for generating Java file-level refactoring candidates. REFINE combines static-analysis-guided smell identification, smell-informed planning, LLM-based transformation, automated re-analysis, preservation checks, and structured reporting. We evaluate REFINE on 450 Java files from 15 open-source systems, producing 1,350 model-pass outputs using OpenAI GPT-5.5, Google Gemini 3.1 Pro Preview, and Anthropic Claude Opus 4.8. REFINE reduces detected code smells by 68.26%, 72.79%, and 68.49% across the three configurations, respectively, with the strongest reductions observed for major smells. A matched 150-file direct-prompt baseline shows that REFINE achieves a higher median code-smell reduction with smaller edits and fewer public-method removals. However, broader quality improvements are inconsistent, and preservation checks reveal residual risks, including assert/fail-call changes and public-method removal. Therefore, REFINE outputs should be treated as refactoring candidates requiring compilation, testing, dependency analysis, and human review before adoption in repository- or system-level settings.

Muhammad Waseem, Aakash Ahmad, Pekka Abrahamsson · 0 citations
#artificial intelligence Open access Aug 2026

memoria.ia: Resolutive Memory — v1.0.0 Release Candidate 1

Memoria.ia v1.0.0-rc1 — Release Candidate 1 Release date: 2026-08-30 Summary v1.0.0-rc1 is the first publication candidate for the Memoria.ia v1 line. It consolidates the validated Resolutive Memory research lineage with the deployable PC/server product layer and the native/mobile runtime path, while keeping post-v1 experimentation isolated from the release candidate. The release architecture remains: application / OFF.IA / agent ↓ Memoria.ia ↓ Resolutive-DB / BDR Memoria.ia owns memory semantics and state. Resolutive-DB owns durable persistence. Optional LLMs are consumers, not the authoritative memory store. Included capabilities persistent local-first memory state; organization and namespace isolation; provenance and authority lineage; conservative HIT / MISS / UNRESOLVED resolution; semantic, episodic, temporal and relation kernels; correction/supersession behavior with preserved lineage; PC/server FastAPI product boundary; Docker/Compose deployment; provider-neutral language-model adapters; metrics and context-selection instrumentation; integrity-checked backup/restore; native production runtime; Android arm64-v8a mobile ABI; durable native BDR persistence and restart recovery; indexed native resolution for large-memory workloads; reproducibility and release metadata gates; official Memoria.ia visual identity assets. Frozen candidate provenance The functional candidate was frozen at: dc73cbcdddfe20e0729e7e6bdea4697f7e8308cd That commit integrated PR #112, which preserved ranking, confidence, provenance policy, ABI and BDR contracts while adding the indexed native resolve lineage. The release branch adds publication metadata, version alignment, release documentation and current branding without importing post-v1 PR #116 runtime behavior. Validation evidence The exact functional lineage used for this release candidate passed the recorded required gates before release preparation: Android mobile ABI: PASS; native production image: PASS; Ubuntu/Windows candidate regression: PASS; BDR Linux/Ubuntu/Windows integration: PASS; native 100 / 1k / 10k benchmark matrix: PASS. Recorded 10k native resolve benchmark improvement versus the prior frozen baseline: p50: 693.233 ms -> 6.288 ms (~110x); p95: 710.630 ms -> 6.391 ms (~111x). These figures are environment- and workload-specific benchmark evidence, not universal latency guarantees. Publication metadata Release version: 1.0.0-rc1 Python package version: 1.0.0rc1 License: Resolutive Research and Non-Commercial License (RRNCL) v1.0 Author: Marcelo Roldão Matos ORCID: 0009-0003-6075-4680 RSMS compatibility: 1.0-rc.1 A new archival DOI should be assigned to this publication. The v0.95 DOI must not be reused as the release DOI for v1.0.0-rc1. Why this is RC1 rather than final v1.0 The repository currently declares compatibility with RSMS 1.0-rc.1, and the published Resolutive Science baseline remains on that release-candidate specification. Therefore Memoria.ia is published as v1.0.0-rc1 rather than claiming final v1.0 compatibility prematurely. Final v1.0 promotion requires: successful release-candidate metadata and regression gates; reproducibility from the public release state; compatibility re-audit against stable RSMS; no release-blocking regression found during RC use; final archival metadata and DOI synchronization. Explicitly excluded from RC1 The following post-v1 work is not part of this release candidate: external/public knowledge learning from OFF.IA Curiosity (issue #114 / PR #116); autonomous curiosity policy; new MA2A federation transport; multimodal post-v1 expansion; new semantic-consolidation phases from the post-v1 roadmap. Those features continue independently after this publication. Security boundary This release candidate is not represented as independently production-security certified. Authentication, isolation, integrity and negative-path controls exist and are tested, but no independent production security audit is claimed. Claims boundary This release does not claim: artificial general intelligence; biological equivalence; replacement of general-purpose LLMs; universal O(1) semantic resolution; production-ready MA2A federation; security certification. Claims are limited to the implementation, tests, benchmarks and reproducible evidence recorded in the repository.

MARCELO ROLDAO MATOS · 0 citations
#software testing Open access Aug 2026

FitGen: Artifact for Validation and Guided Repair of LLM Generated Executable Fitness Functions for Search Based Software Testing

Artifact for the paper “FitGen: Validation and Guided Repair of LLM Generated Executable Fitness Functions for Search Based Software Testing,” published in the Proceedings of the 7th International Workshop on Automated Program Repair (APR '26), co-located with ASE 2026. This artifact contains the source code, benchmark programs, prompts, generated fitness functions, experimental results, and supporting scripts used in the evaluation of FitGen. Detailed setup, dependencies, repository structure, and reproduction instructions are provided in the included README.md file.

Md. Al Muzahid Nayim, Faezeh Rajabi Kouchi, Amit Kumar Sikder et al. · 0 citations
#software testing Dataset Open access Aug 2026

SI-Mamba: Data, code, pretrained models, and numerical results

This record contains the simulated Zernike-coefficient datasets, source code, model configurations, pretrained weights, and machine-readable numerical results supporting the SI-Mamba study. The archived learning models are LSTM, Mamba, and SI-Mamba. It includes the training set, five held-out test sets, 25 boundary-condition datasets, evaluation and boundary-analysis scripts, and the exact software environment. Figure-generation code and figures are intentionally excluded. Source code is licensed under Apache-2.0; data, pretrained weights, and numerical results are licensed under CC BY 4.0. See the included README and SHA256SUMS for reproduction instructions and integrity verification.

qin xiao · 0 citations
#software testing Open access Aug 2026

Code Factory: Proof-by-Sabotage Software Factory

Your AI can say the test passed. Code Factory asks whether the test could ever have failed. Solo developers start with one local proof; teams bind the real diff, intent, independent checks, and receipts instead of trusting an agent's narrative. Version 0.45.1 adds a deterministic AppForge App Review evidence gate: 30 policy and release-risk checks bind the exact app build to required evidence, preserve unknowns, and keep final submission under named human control. The Graph Ops mission-control storyboard turns that review into a visible mission, tension, guidance, agency, transformation, and ready handoff. The workflow is designed to reduce avoidable App Review rework and waiting time; it does not guarantee approval or claim a measured rejection-rate reduction. Supplied local observations are not provider certification, payment settlement, production proof, security or compliance certification, or release authority. Dual licensed under MIT or Apache-2.0.

Richard Katz · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Aug 17, 2026

Q&A: Rethinking how innovation happens

In his latest book, Professor Eugene Fitzgerald examines the forces that turn breakthroughs into value — and why innovation resists simple formulas.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.