Usability Challenges across AI-Assisted Infrastructure, Security, and Scientific Workflows

Abstract

AI-assisted tooling is increasingly being integrated into infrastructure management, security operations, software engineering, and scientific computing environments. While discussion around AI adoption often emphasizes automation and productivity, expert users still need to understand what these systems are doing, decide when to trust their recommendations, and know what to do when the system is wrong or uncertain. This talk synthesizes findings and observations from UX research across enterprise data classification, identity and access management (IAM), backup and recovery, ransomware resilience, and AI-assisted infrastructure workflows. The evidence includes studies with cloud developers, security and compliance practitioners, and IT and site reliability professionals, alongside ongoing design and research work examining how expert users interact with AI-supported technical systems. Across these domains, three recurring usability challenges emerge: unclear system state, difficulty evaluating AI-generated recommendations, and limited support for correcting or recovering from AI-supported actions when something goes wrong. Although the technical environments differ, users repeatedly need to understand what the system observed or changed, evaluate whether a recommendation is appropriate for their specific context, and determine what to verify, reverse, or escalate before taking consequential action. These challenges have direct implications for research software engineering. AI-assisted code generation, workflow scripting, cloud and HPC configuration, notebook assistants, debugging, testing, and scientific automation introduce similar questions about visibility, trust, accountability, and recoverability. Lessons from enterprise infrastructure and security therefore provide useful design considerations for RSEs evaluating or building AI-assisted research software. The session offers a cross-domain synthesis of these recurring human-AI interaction challenges and practical questions for evaluating AI-assisted systems. Attendees will leave with a set of questions they can use to evaluate AI-assisted systems: whether users can understand system state, decide when to rely on a recommendation and when to question it, intervene when necessary, and verify or recover from the resulting actions. Connection to Mission, Goals, & Interests of US-RSE Community This talk aligns closely with the USRSE’26 theme, “Advancing Science in the Age of AI,” by focusing on the human and operational challenges that emerge as AI-assisted tooling is integrated into technical and scientific workflows. My background is in UX research and human factors across enterprise infrastructure, cloud systems, security tooling, and scientific software environments. Across these domains, I have observed the same challenges emerge when expert users interact with AI systems, including issues with transparency, trust calibration, oversight, error recovery, and workflow integration. While conversations around AI often emphasize capability and productivity, this session examines how usability and human-centered design influence the successful adoption of AI within complex environments. These topics are directly relevant to research software engineers who develop, evaluate, maintain, or adopt AI-assisted tools as part of scientific workflows. The goal of this session is to encourage discussion around designing AI systems that better support expert users while maintaining reliability, transparency, and human understanding. These topics connect strongly with the US-RSE community’s broader interests in sustainable, usable, and effective research software.