Skip to content
Conference Open access

A Seven-Criteria Framework to Evaluate Initiatives for Generative AI-Based Solutions

Jul 2026 · Proceedings of the International Conference on Business Excellence · Vol 20, pp. 1599 - 1612 · 0 citations · 13 references

TL;DR

A list of seven evaluation dimensions — domain effectiveness and impact, regulatory compliance and ethical risk, user adoption, business value, technical feasibility, data availability and quality, and competitive differentiation that can be used to characterize applications based on generative artificial intelligence technology are proposed.

Abstract

Abstract The development and potential of large language models created incentives for organizations to adopt and integrate this technology in their internal processes. When the technological layer of organizations is in an acceleration phase there is a strong need to manage this change. Currently, generative artificial intelligence brings a vast area of opportunities, but this can create an “analysis paralysis” when deciding what initiatives to pursue. Different frameworks exist but they are targeting software solutions or specific domains. This paper follows an exploratory, framework-development qualitative design. An evaluation framework to assess initiatives based on generative artificial intelligence technology is developed by reusing existing decision models and enhancing them with criteria based on a simulated focus group using large language model personas. The interim findings were analysed using directed qualitative content analysis (deductive coding to a baseline codebook derived from literature), complemented by qualitative methods (free listing of evaluation criteria, anchored rating-scale elicitation, priority ranking, mapping to baseline criteria). A derived seven-criteria framework with definitions and performance scales resulted. This allows organizations to maximize chances of choosing the highest-reward initiatives and invest resources efficiently when needed to select, implement and integrate generative artificial intelligence solutions. The intention of this paper is to propose a list of seven evaluation dimensions — domain effectiveness and impact, regulatory compliance and ethical risk, user adoption, business value, technical feasibility, data availability and quality, and competitive differentiation that can be used to characterize applications based on this technology. [ms1][ms2]

Read PDF

Similar papers

Conference Jul 2026

AI for Managing Projects: Methodology for Developing Use Cases

The increasing availability of Artificial Intelligence (AI) tools has generated significant interest within the project management community; however, structured guidance tailored specifically to Project Managers remains limited. This paper proposes a standardized AI-enabled prompt architecture aligned with the PMBOK® 8E performance domains and processes. The framework consists of a Master Prompt, Task-Level Executable Prompts, and Refine Prompts designed to create bounded, context-aware interactions between Project Managers and AI systems. The proposed architecture embeds PMBOK-aligned terminology and follows the Inputs–Tools–Outputs (ITTO) logic. The AI system functions as an analytical and generative tool within this structure, processing structured and unstructured inputs—including expert judgment—and producing standardized outputs for managerial review and refinement. The architecture is designed to be generalized and extensible across all forty project management processes. This study does not present empirical performance metrics; rather, it introduces a structured conceptual framework intended for practical application and future validation. Practitioners are encouraged to implement the architecture in real project environments to evaluate measurable improvements and contribute to further academic development in AI-enabled project management.

Vittal Anantamula, Rajendra Harsh · 0 citations
Review Open access Aug 2026

AI between ambitious improvements and accelerated idiocy: a framework for capturing the advantages and disadvantages of AI projects

Artificial intelligence (AI) is changing entire business models and markets, and we are arguably witnessing only the beginning of its impact on business-to-business (B2B) firms and markets. While new technologies and transformations have been core areas in the B2B marketing literature and, thus, theories, models and frameworks for managing digital innovation already exist, the managerial realities AI imposes on organizations are manyfold and potential impacts have arguably been listed in an unstructured, random and eclectic way. The purpose of this paper is to offer a framework structuring advantages and disadvantages of AI. Against this backdrop, the authors conceptualize a framework entailing 10 specific features creating a consolidated overview and reflecting the potentially paradoxical tensions between the advantages and disadvantages inherent in AI projects. The framework makes these tensions explicit and managerially addressable. The authors field tested the framework with 145 executives to judge its relevance and viability. The paper presents a field-tested framework which enables managerial practice and guides further research. The paper presents an inclusive framework that summarizes a wide area of arguments into a unique framework.

C. L. Pedersen, Thomas Ritter · 0 citations
Review Open access Jul 2026

A Taylorian Approach for Managing Generative AI: Lessons from One Year of Collaborative Multi-Industry Workshops

Abstract OVERVIEW: This article explores how “Taylorism,” the historical management approach created by Frederik Winslow Taylor, can be adapted to tackle contemporary management challenges posed by generative artificial intelligence (GenAI). Drawing on a longitudinal and collaborative study based on five multi-industry workshops conducted over one year, we developed a three-dimensional framework for managing GenAI in organizations. This framework offers a three-part action plan for practitioners: (1) establish dedicated GenAI labs to orchestrate experimentation and learning; (2) explore GenAI in a structured, capability-driven manner; and (3) redefine organizational roles and workflows to support generativity. Grounded in historical parallels with Taylor’s scientific management principles, our findings empirically extend existing GenAI management frameworks by revealing how organizations operationalize them. We discuss implications for practice and suggest avenues for future research at the intersection of GenAI, technology management, and organizational innovation. We present actionable insights for managers navigating the integration of GenAI technologies, emphasizing collective actions and cross-functional collaboration. PRACTITIONER TAKEAWAYS Build before you scale: Establish a dedicated GenAI laboratory to coordinate experimentation before scaling use cases across business units. Start with capabilities, not problems: Explore GenAI use cases by mapping the technology’s capabilities to your organizational activities. Slow down to go fast: Invest in reflective functions such as evaluation and governance roles. These may initially seem burdensome, but they are essential to preventing GenAI initiatives from becoming the next “AI winter.”

Antoine Bordas, P. le Masson, B. Weil · 0 citations
Jul 2026

Preparation for incorporating generative artificial intelligence in learning and development programs: a strategic approach

Adoption of generative artificial intelligence (GenAI) technologies has spread rapidly, including in many HR departments. This paper discusses a strategic approach toward implementing Gen AI into learning and development (L&D) programs and courses. This is a “viewpoint” paper from the perspective of two authors, one an academic who researches the impact of AI on HR, the other an executive coach and corporate learning and development consultant with years of experience as a chief learning officer. L&D programs are ultimately about capturing and enhancing organizational knowledge toward gaining competitive advantage. A strategic approach to use of GenAI, including the support of top management, incorporation of the principles of a “learning organization,” consultation of key stakeholders and appropriate choice of initial content development, will enhance the likelihood of success. Appropriate GenAI implementation will address employee engagement and improve the firm’s learning culture. Not all GenAI adoptions have been successful. Organizations that do not take a strategic approach may end up with a failed or suboptimal outcome in their L&D GenAI projects in comparison to those who do.

R. Hamilton, W. Sodeman · 0 citations
Review 2026

From AI Ideas to Institutional Priorities: A Case Study of a Faculty-Led Pipeline for AI-Enhanced Learning

Background: Generative artificial intelligence has accelerated the diversity of proposed educational innovations, creating a practical challenge: how to evaluate, compare, and prioritize AI-enhanced learning ideas transparently. Existing guidance often emphasizes instructor-level decisions, while fewer models address institutional prioritization. Purpose: This case study describes a faculty-led process for identifying, prioritizing, refining, and recommending AI-enhanced learning use cases for leadership consideration. Approach: A teaching and learning AI task force representing seven professional schools developed a five-stage pipeline: structured ideation, transparent scoring and prioritization, deliberation and consensus building, use case refinement, and leadership-facing submission. This workflow supported proposal comparability, disciplined prioritization, and equity review before institutional decision-making. Outcomes: The task force evaluated 24 proposals and 234 scored entries. Bounded AI-assisted analysis supported score synthesis, comment-theme identification, and thematic grouping, informing rather than replacing faculty decision-making. Following deliberation, related high-priority proposals were synthesized into three categories for refinement: course-grounded virtual teaching assistants, simulation practice engines, and faculty augmentation systems. Implications: The pipeline offers a transferable model for managing innovation overload, evaluating innovation potential, and advancing responsible AI-enhanced learning.

Cory Stephens, Shannon Tucker, Eric S. Belt et al. · 0 citations
Book Open access Jul 2026

Engineering Practices in Open-Source Agent Frameworks: A Large-Scale Empirical Study

A large-scale empirical study of mainstream open-source agent frameworks from an engineering perspective, providing empirical evidence linking framework design choices to engineering risks and highlighting the need for stronger guidance and support in agent framework development.

Yibo Zhai, Junjun Si, Yan Wang et al. · 0 citations