Splitting a long agent build into blocks cost 8x more tokens

On 2026-10-22 we built an "anti-rot" controller for Chief, the part of Melta that hands tasks to coding agent sessions. It scores how stale a session's context has become and decides whether to keep going, checkpoint, or start fresh, and it sends any build longer than 8 steps in blocks. Before turning it on, we measured it on a Raspberry Pi. The same 40-step pack read 811K cached tokens in 8-step blocks and 97K as one prompt. So it shipped opt-in and off by default. The rule we took from it: count how many turns a task really takes before you split it, because with prompt caching every extra first turn pays for the whole prefix again.

What did the controller do?

An agent session that runs for a long time collects old file contents, failed attempts and finished subtasks in its context. Past some length the agent gets worse at the task in front of it. That is what people call context rot, and the usual fixes are summaries, checkpoints and sub-sessions.

Chief's version scores each new task against the session it might go to. The score rises when the task touches other files than the session's recent work, when the context is deep, when there have been errors in a row, when the topic has shifted, and when the cache is cold. Then it picks one of three outcomes:

• Append. Send the task to the open session as it is.
• Checkpoint. Write a handoff of at most 2K tokens (state, files touched, next step), clear the context, and start one fresh first turn with the handoff and the next work.
• New session. Start a new session from the handoff.

Long builds got the same treatment. A pack with more than 8 numbered steps went in blocks of 8, with a checkpoint between blocks, so no single stretch of work could grow a long context.

The decision

Here is the decision as we would write it now. The first block, the turn-count guard, is the part we learned:

# measure first: median turns for
# a pack like this, this worker
turns = median_turns(pack, worker)
if turns <= ROT_TURNS:
    # never reaches the point
    # where quality falls
    send pack as one prompt; append
    return

# 0-100: file overlap, context depth,
# errors in a row, topic shift,
# cold cache
score = rot_score(task, session)
if score < 35:
    append
elif score <= 65:
    # handoff <= 2K, clear,
    # one fresh first turn
    checkpoint
else:
    new session from the handoff

if steps > 8:
    send in blocks of 8,
    checkpoint between blocks

To be plain about it: the guard is not in what shipped. Chief has the rot score and the blocks, behind the opt-in switch. The guard is the rule the measurement taught us, and ROT_TURNS has no measured value yet.

What did we measure?

We ran two 40-step packs both ways on the same machine with the current worker model, and summed the cached tokens read across every segment of each build:

Both builds landed either way. Splitting did not make the work better; it made it cost about 8x more on the doc-notes pack and about 4x more on the code pack.

Why did splitting cost more?

Because of how prompt caching prices a turn. Every turn sends the whole prefix again: the system prompt, the tool definitions and the pack. With caching, that prefix is read from cache instead of processed fresh, which is cheaper per token, but it is still read in full on every turn.

As one prompt, the worker read that prefix 3 times for the doc notes and 6 times for the code. It finished the whole pack in those few turns, so its context stayed short and never reached the length where rot sets in. There was nothing for a checkpoint to fix.

In blocks, each block started with its own fresh first turn over the same prefix, and then spent its own round of turns: edit, test, commit, report, write the handoff. Every block added one of those rounds, each paying the prefix several times.

So the price of a checkpoint is easy to write down: one new first turn, times the size of the prefix, plus the turns to write and read the handoff. It pays back only when the agent would otherwise run long enough for its context to grow well past that, and for quality to fall.

Does this apply to your harness?

Probably. Summaries, checkpoints and sub-sessions all share the shape: they shorten the context by starting a new conversation, and every new conversation pays its first turn again. That cost is fixed per split, while the benefit depends on how long the agent would have run. A harness that splits by step count, not by measured turns, can pay the fixed cost on every task and collect the benefit on none.

The design was plausible. Long builds do rot; a 2K handoff is small; blocks of 8 sound careful. Only the measurement showed that the worker we have today finishes too fast for any of it to matter. It is the same family as our post on cost estimates, except that this time the measurement overturned the design before it went on.

Checklist

• Count turns per task first.
• Price a checkpoint as a new first turn times the prefix. Add the turns spent writing and reading the handoff.
• Split only above the turn count where quality falls. Below it, one prompt is cheaper and no worse.
• Keep the switch opt-in until 20 measured tasks show a saving. A design that sounds careful is not a result.
• Record cached and uncached tokens separately. Cache reads are where splitting hides its cost.

Common questions

What is context rot?

It is the drop in an agent's quality as its context fills with old material from a long session.

Do summaries or checkpoints save tokens?

Only when they replace a long run. Each one starts a new first turn that re-reads the cached prefix, so on short tasks they add tokens.

Melta is free; join the waitlist.

Sources

• Melta docs: Chief routing
• Your agent's cost estimate is not a measurement