Skip to content

DynBranch: Speculative Subgraph Reuse for Dynamic Agentic LLM Serving

Sep 2026 · 0 citations · 112 references
Computer Science

TL;DR

DynBranch is proposed, which makes an unresolved branch addressable before it resolves, and its stable coordinate lets candidate subgraphs run during resolution and completed subgraph results be reused across later requests.

Abstract

Agentic LLM workflows decide their execution paths at runtime. Downstream computation may be predictable, or may have run before, yet it cannot begin until the model or the user resolves the branch. We call this serialization the branch-resolution barrier. Caching alone does not hide it: the key that identifies a reusable result is not known until then. In this paper, we propose DynBranch, which makes an unresolved branch addressable before it resolves. Its stable coordinate lets candidate subgraphs run during resolution and completed subgraph results be reused across later requests. A two-level controller admits this work when its expected benefit exceeds the load price. DynBranch sits at the model-API boundary and requires no changes to agent harnesses or model execution engines. Across four agentic workloads with Qwen3-32B on 4x H200 GPUs, DynBranch reduces mean latency by up to 32% over each workload's strongest prior system and by 46-66% against a no-reuse floor, while preserving workflow results. The benefit persists across backbone families and on a commodity Qwen3-8B/RTX 4090 deployment.

View source

Similar papers

Sep 2026

PipeSwift: Revisiting Pipeline Parallelism for Large-Scale Completion-Oriented Agentic LLM Serving

It is shown that pipeline parallelism (PP), long overlooked because it offers little decode-latency advantage, can reduce JCT by providing a more favorable balance between prefill and decode efficiency, and PipeSwift is built, an optimized open-source pipeline-parallel runtime integrated with a tailored micro-batch par...

Shiju Wang, Fei Ren, Fang-Cheng Fu et al. · 0 citations
#artificial intelligence Preprint Sep 2026

LazyAgent: Demand-Driven Materialization and Physical Optimization of Agentic Programs

LazyAgent is presented, a unified execution framework for agent-authored programs organized around a live, goal-derived demanded set that consistently outperforms the strongest goal-stopping eager baseline by refusing unrelated work before it starts and motivates benchmarks built from standing programs and sequences.

Xin Heng · 0 citations
Preprint Sep 2026

AKTS: Sub-Microsecond Kernel Policy Switching for Language-Model Agents

AKTS, which verifies a policy library once, at load time, and reduces the agent's runtime action to writing an integer index into an in-kernel array of preverified policies, and an in-kernel tail call resolves that index.

Mohammadali Khodabandehlou, Mahdi Alizadeh · 0 citations
Preprint Aug 2026

PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents

PeakBench is a benchmark of executable multi-tool workflows with execution-grounded dependency annotations and measured resource profiles that shows that strong logical planning does not reliably translate into safe or efficient execution under resource constraints, and exposes resource information to reduce avoidable...

Zhi-Kai Chen, Xu-Xiang Zhong, Song-Yan Li et al. · 2 citations
#machine learning Preprint Sep 2026

EfficientAgent: What Makes KV Cache Offloading Work for Concurrent Agents?

EfficientAgent is presented, which sizes and manages the host tier by this working set, which estimates the working set from agent histories to size the host tier and its predictions, made before the experiments, located the capacity at which offloading starts to pay.

Kun-Ming Shao, Jie-Run Chen, Jiang-Nan Yu et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.