PREreview of "TurnCost: Turns, Not Observations: Cache-Aware Loop Engineering for Coding Agents"

Abstract

This Zenodo record is a permanently preserved version of a PREreview. You can view the complete PREreview at https://prereview.org/reviews/23423668. TurnCost re-scores 600 public coding-agent trajectories (nvidia/Open-SWE-Traces; OpenHands and SWE-agent crossed with Qwen3.5-122B-A10B and Minimax-M2.5, 150 per cell) under an exact prefix-cache simulation (Section 4). The authors find that 98.1% to 99.0% of input tokens come from the cache, so flat pricing overstates cost by 7.1x on average under near-ideal caching (Section 5.3). They also find that turn count alone explains R2 = 0.844 of log trajectory cost, while mean observation size explains 0.040 (Section 5.6). And they find that truncation with recovery guidance is followed by a narrower query in 76.9% of next actions, against 44.0% after a bare marker (Section 5.8). I liked that the truncation baseline B3 sits next to delegation R1 and strong-model compression B2, so routing and compression are not mixed up (Section 4), and that TurnCost, code and derived results are on Zenodo. I read it as a practitioner who runs coding agents every day. In my company the software development is done by AI agents, so maybe my practical side is useful here. Four comments. 1. On where the money sits (Section 5.6, Limitations) - the corpus excludes runs that hit the iteration limit or were cut by the harness, which truncates the right tail of turn counts. In my measurement of 722 agent sessions (150,902 model calls, DOI 10.5281/zenodo.22759216) more than 85% of the modeled cost was work with context. About 3% of sessions gave 80% of the cost, and sessions longer than 200 calls (8% of sessions) gave more than 90% of the money. So the long tail is where the money is, and the filter removes it. Could you re-score a corpus that keeps those runs, or at least report how many were dropped? 2. On the cache (Section 5.3) - the paper finds 98.1% to 99.0% of input tokens served from the prefix cache, and asks that the 7.1x be cited with the near-ideal caveat. In a separate run of 168 sessions, 94.4% of billed tokens were re-reading context already sent (contextburn, DOI 10.5281/zenodo.22712985). So I agree the cache has to be priced first. My question is about checking the simulation. Can TurnCost take real provider usage logs (cache reads, cache writes) where they exist, so the near-ideal figure can be compared with what was actually billed? Block rounding and eviction are listed as not modelled (Limitations), and billed logs would include both. 3. On turns as a knob (Section 6) - the paper asks why an agent needs a hundred turns to fix one bug. In my experiment of 36 coding-agent runs (six context-clearing policies x six repeats, DOI 10.5281/zenodo.22759217), clearing after every task cost about a third more (+33.5%) than clearing every three tasks; never clearing was not significantly different. So a choice about the loop structure, not observation size, moved cost by a third. I also hand heavy reading to a subagent so the main conversation stays clean, which is out-of-band delegation in your terms. Could follow-up work treat when to clear or compact as a turn-axis variable? 4. On which turns are redundant (Section 5.2 Table 2, Section 6) - read/exec turns are 67.4% of turns, and the paper does not show that turns can be removed while keeping task success. In my guides I write that every task needs its own check the agent can run; without it "looks done" is the only signal the agent has. Can you split read/exec turns into test runs and exploration, and see which kind grows in the hundred-turn trajectories? That would point to the turns that are candidates for removal. My own 36-run data is public on Hugging Face (DOI 10.57967/hf/10366), so anyone can recompute it, and I welcome the same openness here. Thank you for a clear and useful paper. Competing interests I cite my own published reports and data in this review (DOIs 10.5281/zenodo.22759216, 10.5281/zenodo.22712985, 10.5281/zenodo.22759217 and 10.57967/hf/10366). I have no relationship with the authors or their institution. Use of Artificial Intelligence (AI) The author declares that they used generative AI to come up with new ideas for their review.