This paper presents a hierarchical, dynamic architecture and software to discover resources across diverse cloud, edge, and HPC systems and exemplifies the importance of careful coordination between agents, discovery tools, and infrastructure for agentic science.
Abstract
Agentic science is transforming the landscape of computational work, extending to scientific pipelines and workload managers. The workloads require specialized hardware within and across institutions. If assessing workload needs against environments is required for scheduling, automated discovery of resources is an essential step. In this paper, we present a hierarchical, dynamic architecture and software to discover resources across diverse cloud, edge, and HPC systems. The design enables concurrent, asynchronous negotiation, selection, and dispatch of requests for work using secretary agents. The agents probe and discover 51 real and simulated providers across 7 categories. We perform 19,973 negotiation and 6,952 selection simulations to assess reliability of decisions, demonstrating high (87.71\%) negotiation accuracy and selection costs comparable to more traditional strategies. Designed for extensibility and currently supporting the Genesis Mission, this architecture exemplifies the importance of careful coordination between agents, discovery tools, and infrastructure for agentic science.
Kiso is situated at the intersection of scientific workflow management and complex, agent-based computing, highlighting its potential to accelerate research on adaptive, self-organizing cyber-physical systems—an emerging frontier in complex systems science.
R. Mayani, K. Vahi, M. Rynge et al.· Frontiers in Complex Systems· 1 citation
The means to execute and orchestrate software components has changed from human-written code to descriptive prose. In high performance computing, this transition is represented in application orchestration, workload management, and system monitoring and debugging, to name a few. The underlying means to enable descriptive definition of tasks is the use of the Large Language Model with associated tool functions and resources. A combination of a model with access to such resources, modeled in software, encompasses an autonomous framework. As fully automated and agentic frameworks are developed for science, it is important to assess reliability and strategies scoped to specific tasks. In this work, we assess the extent to which an agentic framework can optimize and run an HPC scaling study with a low latency network in Amazon Web Services, accurately transform HPC job specifications between workload managers, and design and run an entire biosciences workflow. We find that the framework completes all three tasks while surfacing task-specific failure modes. In the scaling study, agents deploy and optimize applications but monitor running jobs inefficiently, preferring conservative fixed waits over event subscriptions. In job translation, they convert specifications between Slurm and Flux with high accuracy, with processor-affinity flags the most common error. In the bioscience workflow, the agent reproduces an expert-written variant-calling pipeline almost exactly -- agreeing with the reference call set in 18 of 19 completed runs -- and reaches this result through many distinct yet functionally equivalent workflow implementations. This information is invaluable moving forward to developing multi-cluster setups with scheduling and transformation handled by agents.
Agents powered by AI/ML are becoming ingrained in orchestration. Dispatch of work is the task of receiving a request, transforming it for a workload manager, and successfully submitting it. Running scientific workflows across multi-cluster environments introduces substantial challenges of dynamic job transformation, dispatch, and submission to heterogeneous clusters. These tasks are well-suited to agents, which can receive textual instructions for work, prepare job specifications, and dispatch. In this work, we assess the reliability of a dispatch agent across 432 runs, testing all possible combinations of five feature dimensions across four prompt styles. The agent is highly reliable (97.9% success). We test a full orchestration to submit, queue, match, score, select, transform, and dispatch in a multi-cluster experiment. We find that descriptive metadata increases successful execution from 48% to 87% of 220 submitted jobs, eliminating architecture mismatch, and improving performance for five of ten measurable applications by up to 3.3x.
Agentic cloud management is emerging as a practice to automate laborious operations, minimize toil, and improve responsiveness. Despite the rapid development of autonomous management agents, we argue that the fundamental missing piece is a systems foundation to enable safe, effective operations across agents and between agents and human operators. In this paper, we advocate for the need of such a systems foundation and share our efforts on developing CloudWeaver, an agentic management substrate that works across existing cloud-user interfaces and future agent-native interfaces. Specifically, we discuss how CloudWeaver (1) scopes the context of individual agent sessions with local views of cloud resources and (2) coordinates concurrent management operations on shared cloud resources. CloudWeaver offers strong safety guarantees and attributable feedback in the presence of conflicting intents, while preserving concurrency between independent operations. We validate CloudWeaver using a representative Azure API workload.
Minghao Li, Ziqian Liu, Ziyu Mao et al.· 0 citations
Large language models have evolved from curiosity to co-pilot in under four years. With the emergence of agentic AI systems that reason, plan, and execute multi-step tasks autonomously, HPC centers face a new category of user need: researchers expect these tools to be supported, not just permitted. This paper offers a practitioner’s perspective from Purdue’s Rosen Center for Advanced Computing (RCAC), where we have begun deploying system-wide configurations, custom MCP servers, and user guidance for agentic tools. As a demonstration, every word of this manuscript was produced through an agent-first workflow: over one hundred commits of iterative collaboration between human authors and AI agents, documented in a public GitHub repository. We argue that proactive engagement, not prohibition, is the path forward for facilitators who wish to remain relevant in the agentic era.
Geoffery Lentner, Ashish· Practice and Experience in A...· 0 citations
The main conclusion is that practical Agentic IoT depends less on placing an entire agent at one tier than on partitioning perception, memory, reasoning, and action under explicit latency, privacy, reliability, and safety constraints.