Aug 2026· European Conference on Knowledge Management· Vol 27, pp. 1461-1470· 0 citations· 18 references
TL;DR
A practical framework for organizations introducing API performance testing from scratch and a knowledge management framing that treats performance testing outputs as reusable organizational assets are contributed.
Abstract
API performance testing is often introduced informally and remains dependent on individual expertise, which limits reuse and makes results difficult to repeat. This paper presents a lightweight framework and repository approach. It helps organizations establish a repeatable API performance testing process and capture knowledge for reuse. Using a Design Science Research (DSR) approach, a Knowledge-Centric API Performance Testing Framework with six components was developed: Scope & SLOs, Environment Setup, Data Preparation, Execution & Metrics, Analysis & Reporting, and Knowledge Capture & Sharing. A key distinguishing feature of the framework is its explicit inclusion of non-technical roles (e.g., Business Analysts and Product Owners), enabling them to contribute to performance testing through structured involvement in scope definition, governance, and knowledge capture. The framework was applied in a global IT organization during an initiative to validate API performance after a data source change. The case shows that a structured workflow and shared artefacts reduce repeated effort and support cross-functional collaboration. The paper contributes (1) a practical framework for organizations introducing API performance testing from scratch and (2) a knowledge management framing that treats performance testing outputs as reusable organizational assets.
DevCoach is presented as a reference architecture built on the Model Context Protocol to make GenAI assistance more lifecycle-aware, governable, and inspectable in organizational settings and reframes GenAI integration as a software engineering design and governance problem rather than a prompt-level optimization problem.
A lifecycle-aware framework that integrates quantitative software quality assessment with Large Language Model (LLM)-based code refinement is proposed and the potential of metric-driven LLM feedback for research software quality improvement is demonstrated while highlighting its inherently multi-objective nature.
Nafis Tanveer Islam, N. Soveizi, Yutong Li et al.· 0 citations
User experience (UX) evaluation is central to software quality, yet it remains difficult to integrate into software engineering workflows in a systematic, explainable, and early-stage manner. This paper presents TraceUX, a framework for operationalizing UX heuristics and design guidance into a rule-based static evaluation pipeline that combines machine-interpretable formalization, executability-aware assessment, context-sensitive scoring, and actionable reporting. The framework is instantiated using Apple Human Interface Guidelines, Swift abstract syntax trees, and mobile games, and implemented in a proof-of-concept tool named TraceHIG. Evaluation was conducted in four layers: analysis of the full rule repository, controlled synthetic validation with injected violations, baseline assessment of 12 public Swift game projects, and a case study on one project. The full repository contained 206 rules; after excluding non-iOS yet platform-specific rules, 193 rules were retained for the downstream experiments. In controlled validation, 216 injected violations yielded 99.2% precision, 61.6% recall, and an F1-score of 0.760. In baseline analysis, overall project scores ranged from 41.6 to 88.0, reflecting rule-conformance spread under the instantiated rule base rather than direct measures of UX quality. The case study demonstrated that profile-aware scoring can yield materially different UX assessments for the same codebase under different game configurations, highlighting the importance of app profiling in static UX evaluation. These findings show that a meaningful subset of UX knowledge can be operationalized into explainable, context-aware static analysis that provides structured and actionable decision support while complementing, rather than replacing, manual and empirical UX evaluation.
Understanding large, complex software architectures is difficult and time-consuming. Software architecture recovery aims to extract explanatory artefacts from code, but existing methods often lack generalisability. To address this, we previously proposed Deductive Software Architecture Recovery (DSAR) to extend current techniques. This paper presents a qualitative evaluation of DSAR in an industrial setting. Using a large language model-assisted prototype, we examined the recovered architecture's content quality, presentation quality, and usefulness for understanding and maintaining real-world systems. To ensure practical relevance, we conducted an in-depth judgement study with software engineering teams at ASML applying DSAR to 17 Java repositories across diverse teams. We contribute: an empirical assessment indicating that DSAR supports comprehension, architectural reasoning, and communication; qualitative evidence on the strengths and limitations of the generated explanatory artefacts; and recommendations for practitioners and researchers on adopting and further developing DSAR for architectural sense-making.
S. A. Rukmono, Lina Ochoa, Thijs Bressers et al.· SIGSOFT FSE Companion· 0 citations
Automated Compliance Checking (ACC) systems are fundamentally static, unable to easily adapt to new regulations, project constraints, organizational, or practitioner-defined rules. This paper presents a framework integrating Knowledge Graphs (KGs) and Large Language Models (LLMs) to support a more extensible design review environment. In this framework, the KG acts as a structured repository for rules and executable logic, while the LLM serves as an intelligent interface. The central innovation is the human-in-the-loop feedback mechanism, where new logic generated by the LLM is validated, executed, and permanently stored in the KG, transforming it into an active, evolving validation engine. Following a Design Science Research (DSR) methodology, we implement and evaluate the framework as a prototype embedded as an Autodesk Revit add-in, demonstrating its ability to retrieve and execute existing rules from the KG, capture new requests during design, and maintain a verifiable, adaptive compliance checking system. Across a two-experiment evaluation, the system achieved 100% mapping accuracy for six existing rules, while generating new executable rules from natural language succeeded in 70% of 20 trials. Performance was strong on parameter-based checks (100%) but dropped on rules involving spatial reasoning (20–60%), where the LLM still struggles to produce reliable logic.
Maen Alnuzha, Tanya Bloch· Journal of Information Techn...· 1 citation
Collaborative AI experimentation across industry and academia requires platforms that enable rapid prototyping while preserving controlled access, tenant separation, and transparent workflows. Despite growing interest in AI sandboxes, there is still limited practical guidance on how to design and implement platforms that integrate experimentation capabilities with governance requirements. This work presents the design and implementation of a governance-aware, multi-tenant AI sandbox for structured experimentation and the generation of reusable evaluation evidence across projects and stakeholder groups. The sandbox was developed within an industry-academia collaboration based on requirements that were iteratively refined with industrial partners. Its reference architecture separates the multi-tenant user interface from the backend control plane and places execution and data-management functions in dedicated layers. The platform supports governed user onboarding, project-centered collaboration, managed access to AI services, approval workflows, audit logging, and traceable experimentation. Experiment configurations, contextual information, and governance decisions are stored as persistent records, allowing evidence and outcomes to be compared and reused across projects. The development process provides practical lessons for deploying and extending governance-aware AI sandbox platforms in collaborative research and industrial environments.
Muhammad Waseem, M. Islam, Md Nasir Uddin Shuvo et al.· 0 citations