Skip to content

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

Sep 2026 · arXiv.org · Vol abs/2609.38201 · 0 citations · 37 references
Computer Science

TL;DR

TomasuLLM is presented, a runtime that executes agent tool calls out of trajectory order while preserving task-execution correctness, and improves the reported benchmark means and scales with tool latency.

Abstract

Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. This observation stall presents the same tension that drove out-of-order processors -- asequential interface hides work that can be predicted and started early, but a speculative result may become visible only after it and every earlier step have been validated. We present TomasuLLM, a runtime that executes agent tool calls out of trajectory order while preserving task-execution correctness. It drafts future actions, runs them in isolated copy-on-write sandboxes, traces their dependencies and effects, and commits results in trajectory order only after validation against committed state. Across three benchmarks spanning sub-second to minutes-long tool calls, TomasuLLM improves the reported benchmark means and scales with tool latency: 1.31x on 100 SWE-bench Verified tasks, 1.35x on 28 Terminal-Bench 2.0 tasks, and 1.27x matched progress on 18 SWE-Marathon sessions. Across 4,010 audited commit-validation records, it produces zero false accepts.

View source

Similar papers

Preprint Sep 2026

JustFit: Just-in-Time State Management for Local LLM Serving

JustFit is presented, an MLX runtime that coordinates their overlapping allocations: KVExec executes and checkpoints four-bit KV with bounded workspace, PhaseSwap loads phase-dependent components, and StateTrans preserves history across execution modes.

Yu-Hua Chen · 0 citations
Preprint Sep 2026

UNISON: A Co-Designed Near-Memory Scheduler of Session KV Residency for LLM Agents

This work presents Unified Native Inter-turn Session Orchestration Nexus (UNISON), an event-driven near-memory scheduler in which Survival-Penalty Eviction for Agent Return-gap and Tiering in Idle-window DMA Events share one live ranking.

Fan He, Yan Li, Xiao-Yang Zeng · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.