A student survey study is presented that examines perceptions of LLM output understanding, validation effort, trust and the perceived usefulness of vibe modeling across several AI-assisted development scenarios to inform future studies for trustworthy and explainable AI-based software engineering via vibe modeling.
Abstract
Large Language Models (LLMs) are increasingly used to generate software artifacts from natural language prompts. While this enables rapid prototyping and lowers the barrier to software creation, it also introduces challenges related to understanding, validation, traceability, and trust. In this paper, we argue that current AI-based development practices focus too heavily on the direct generation of code and insufficiently on intermediate representations that preserve human intent and support reasoning about system behavior. We argue for vibe modeling as a lightweight intermediate abstraction between natural language interaction and code generation. To explore its potential, we present a student survey study that examines perceptions of LLM output understanding, validation effort, trust and the perceived usefulness of vibe modeling across several AI-assisted development scenarios. Our results are intended to inform future studies for trustworthy and explainable AI-based software engineering via vibe modeling.
This research aims to bridge the gap by providing a comprehensive evaluation methodology for LLM-powered agents that is grounded in real-world software development practice and focuses on contamination-awareness, in thewild agentic behavior assessment, and trajectory-aware benchmarks and metrics.
The growing adoption of Large Language Models (LLMs) in Software Engineering has reinforced the expectation that coding activities can be largely automated. However, this perception may represent yet another historical search for a solution capable of eliminating the inherent challenges of software development. This article discusses the transition from a code-centered paradigm to Specification-Driven Development. We argue that artificial intelligence reduces some of the effort associated with writing source code, but it does not eliminate the complexity of developing professional software systems. Instead, it shifts this complexity toward domain understanding, requirements elicitation, specification development, validation, maintenance, and software evolution. Building on this perspective, we discuss the renewed centrality of Requirements Engineering, considering its implications for productivity and software quality, as well as risks associated with automation bias, ambiguity propagation, Specification Overfitting, and the accumulation of Specification Debt. Finally, we propose the Specification Paradox: the more capable artificial intelligence systems become at automatically generating software, the greater the dependence on correct, complete, verifiable, and explainable human-produced specifications. We conclude that the future of Software Engineering will depend not only on machines'ability to generate code, but also on humans'ability to correctly specify, evaluate, and evolve what is intended to be built.
Research at the intersection of human factors analysis and large language models (LLMs) has grown rapidly in recent years; however, much of this work emphasizes automation and efficiency, evaluating success primarily through model-centric metrics. In contrast, this study reframes generative AI not as an automation tool for analysis but as a collaborative partner for cognitive stimulation, and proposes PromptWeave, a prompt-design methodology intended to expand, deepen, and transform an analyst’s reasoning. We applied PromptWeave to industrial accident cases and conducted a quantitative evaluation using human-centered KPIs. The results indicate consistently high performance across all KPIs, supporting the utility of PromptWeave as a reproducible collaboration protocol executable on an LLM platform.
Software effort estimation remains a cornerstone of project planning and control, yet existing estimation models are grounded in the assumption that software development effort is dominated by human reasoning and manual construction. The rapid integration of large language models (LLMs) into development workflows fundamentally challenges this assumption by automating substantial portions of code synthesis while shifting human effort toward supervision, validation, and integration. As a result, traditional effort estimation proxies such as Story Points and size-based metrics may no longer reliably characterize development effort. This paper presents an empirical study examining how effort manifests in LLM-assisted software development. Rather than using LLMs as predictive estimation tools, we investigate how their adoption reshapes the underlying cost structure of development work. We introduce the notion of Hybrid Intelligence Effort (HIE), conceptualizing effort as the combined burden of model-performed reasoning and human oversight activities. Using a controlled experiment involving 22 developers, 110 real-world tasks, and three LLMs, we compare the explanatory power of traditional estimation metrics against interaction- and oversight-based Hybrid Intelligence dimensions. Our results show that while Story Points retain partial explanatory validity, they fail to capture dominant sources of effort in LLM-assisted workflows. In controlled experiments, HIE dimensions increase explained variance in observed effort from approximately 72–80%, while substantially reducing systematic estimation error. Human validation and corrective intervention emerge as the primary drivers of effort, outweighing artifact-level characteristics. These findings suggest that effort estimation models must move beyond human-centric and size-based assumptions to remain effective in AI-augmented software engineering.
Feisal Alaswad, E. Poovammal, Kadiyala Ramana et al.· Discover Computing· 1 citation· ⚡1
The AI-Native Manifesto envisions large-scale agile software development in which humans increasingly govern intent, risk, and exceptions while agents execute more of the engineering process. Realizing that end-state requires more than better code generation: it requires assurance closure, meaning that the system can establish what must be true, determine and obtain appropriate evidence, judge the credibility of that evidence, preserve its validity through change, and use the resulting uncertainty to bound agent authority. Existing work already provides many of the necessary mechanisms across formal methods, testing, simulation, assurance cases, digital twins, and runtime assurance. We identify six residual gaps in making the surrounding assurance reasoning sufficiently machine-operable, propose a high-level architecture with six corresponding capabilities built on a shared semantic assurance layer, and formulate four research questions to turn that architecture into dependable, human-on-the-loop, AI-native R&D.
Enterprise software requires specification governance to transform probabilistic AI generation into deterministic, auditable engineering, and the SGRM framework is introduced, which defines four-component specification contracts, constrains stochastic generation via deterministic validation, and integrates generation, verification, and governance into a closed-loop architecture.