Dynamic Dialogue for Organizational Knowledge Acquisition: Expert Insights on Agentic AI use for AI Readiness Assessment

Abstract

Organizations increasingly recognize Artificial Intelligence (AI) as a strategic capability, yet many struggle to assess and understand their true readiness to adopt and scale AI solutions. Existing AI readiness assessments rely mostly on static surveys and internally collected data, which often fail to capture organizational dynamics and cross-functional dependencies. They also impose substantial time demands while delivering limited perceived value to participants, particularly in Small Medium Business (SMB) companies. Consequently, assessment outcomes frequently generate fragmented insights, limiting their usefulness for strategic forecasting and organizational learning. This study builds on conceptual framework proposing multi-agent conversational diagnostic system, which combines dynamic interview dialogues and public data (OSINT) to acquire richer knowledge about AI readiness and generate actionable recommendations for AI implementation. To refine and validate the proposed approach, we conducted 15 semi-structured expert interviews with senior practitioners involved in AI transformation, including CEOs, sales and technical directors. The sample represents a balanced cross-section of large enterprises and SMBs. The interviews explored experts’ perceptions of current knowledge management practices in organizational assessments and expectations toward agentic AI systems capable of dynamic dialogue based diagnostics and recommendations. Results indicate expert agreement that static interview and survey approaches inadequately capture contextual organizational knowledge. Experts expressed interest in dynamic dialogue mechanisms that enable clarification and contextual probing to improve knowledge acquisition processes. At the same time, participants highlighted risks associated with agentic AI usage, including concerns regarding diagnostic quality, replicability, transparency and perceived value for participants. Experts further emphasized that integrating internal organizational knowledge with external public data can improve the results. The study contributes to Knowledge Management by reframing AI readiness assessment as a dynamic knowledge acquisition rather than a static task. Using a Design Science Research approach, the proposed agentic AI artefact offers a foundation for developing adaptive diagnostic systems that support strategic decision-making process, enhance organizational learning and improve the actionability of AI readiness assessments.