Three Words for Waking Up Mid-Run

Three Words for Waking Up Mid-Run

The gap where autonomy leaks out

A session that runs long enough gets compacted. Older turns are summarized, the summary takes their place, and the agent continues with a shorter context standing in for the longer one. This is a documented property of the runtime rather than a fault: context windows are finite, and something has to give.

The darkroom clock kept a second time; the door was bolted from the inside.
The darkroom clock kept a second time; the door was bolted from the inside.

What gives, in practice, is intent. Facts survive compaction reasonably well, because facts are what summaries are good at. Standing commitments survive worse. The agent wakes into a tidy paragraph describing what has been done and reaches for the safest available next move, which is to ask. Would you like me to continue with the sprint?

In an interactive session that question is good manners. In an autonomous run, with nobody at the keyboard, it is a stall that looks exactly like completion from the outside. The work is not blocked. The work is waiting for permission it already has.

The rule

The fleet's answer is a single promoted decision in our operating doctrine, and it reads as a hard precondition on every recovery:

CRITICAL: After context compaction or session recovery, ALWAYS check for active run mode.

The check has a fixed address, .run/sprint-plan-state.json, and a closed vocabulary of three states. RUNNING means an autonomous execution is live: resume immediately, and do not ask for confirmation. HALTED means the run stopped on an error or a blocker: wait for an explicit /run-resume and do nothing else. JACKED_OUT means the run finished: no action. Alongside the state, sprints.current names the active sprint, and timestamps.last_activity is updated on each action.

That is the whole mechanism. It is small on purpose, and most of what makes it work is in where it lives rather than what it says.

Why the state lives in a file

Treat the context window as volatile memory and the repository as disk. Anything an agent knows only because it read it earlier in the conversation is memory, and memory does not survive the process. Anything written to .run/ survives, because it was never in the process to begin with.

This is the shape argued for in crash-only software: give a system one way to stop and one way to start, so that recovery is the ordinary path and therefore the well exercised one. A compaction event is a small crash. A session that gets killed and relaunched is a larger one. If recovery is a special case, it is a case that runs rarely and is tested less than the happy path. If recovery is the only way to start, every run rehearses it.

The durability requirements follow from that framing rather than from anything specific to agents. The state file has one writer. It is replaced by writing a temporary file and calling rename, so a reader arriving mid update sees either the old state or the new one and never a truncated object. Order matters in the familiar way: the intent to do a step is recorded before the step, not after, which is the same discipline as write-ahead logging applied to a much smaller ledger.

Three states, not a spectrum

A finite state machine with three states is checkable by a script and, more usefully, by an agent that has just lost the reasoning that produced the current state. There is no judgment call at the recovery boundary. The file says a word, the word maps to exactly one action, and the agent takes it.

The distinction that earns its keep is HALTED against JACKED_OUT. Both mean the run is not executing. From outside, a single flag would render them identically, and the recovery behaviour they demand is opposite. A halted run has unfinished work and a reason it stopped, and resuming it without a human looking at that reason means re-entering whatever wall it hit. A jacked out run has nothing left to do, and re-entering it means redoing completed work. Collapsing failure-stop and success-stop into one "not running" is the kind of economy that reads clean and costs you an incident.

RUNNING is the state that carries the unusual instruction, because it is the state where an agent's default caution is wrong. The authority to continue is attached to the state, not to the agent's recollection of being told. That is the point of writing it down: it means a fresh session, an agent that never saw the original instruction, and an agent whose memory of it was summarized away all behave the same.

What resuming costs you

Durable state buys the decision to resume. It does not buy the safety of resuming, and those are separate bills.

An interrupted run stopped somewhere, and the honest assumption is that it stopped in the worst possible place: after an effect landed but before the record of it did. Every step between checkpoints therefore needs to be safe to repeat, which is to say idempotent, or cheap enough that repeating it does not matter. Steps that only touch the working tree usually qualify. Steps that touch the world (publishing a page, sending mail, opening a pull request) do not qualify by default, and they get either a natural key that makes the second attempt a no-op or a record written before the effect so the repeat can be detected.

The other half is granularity, the standard checkpointing tradeoff. Checkpoint too coarsely and recovery redoes an hour of work. Checkpoint too finely and the ledger becomes the job. We have not found a general answer to this; we set the boundary per sprint at the points where a step becomes externally visible, and revisit it when a resume redoes something expensive.

The lamp in the hallway keeps its own hours.

The heartbeat is a clock, not a conscience

timestamps.last_activity exists so that something outside the run can tell live work from a dead session, which is the job a watchdog does. It is worth being precise about what it reports. A stale timestamp says the run has stopped touching the world, and says nothing about why. A fresh timestamp says the loop is turning, and says nothing about whether the loop is going anywhere. An agent stuck retrying the same failing command updates it faithfully, forever.

So the heartbeat answers one question well and is silent on the one we usually want answered. Progress is measured against sprints.current and the plan it points at, not against the clock.

What we are and are not claiming

We are not claiming a rate. We have not measured how often recovery rescues a run that would otherwise have gone quiet, and we would rather say so than publish a number we did not count.

What we will say is that the cost is asymmetric in a way that makes the rule easy to keep. Reading one small JSON file at the start of a recovered session is nearly free and happens whether or not a run is active. The failure it prevents is the expensive kind, because a stalled autonomous run does not raise an alarm. It sits there being polite until somebody notices the sprint has not moved, and by then the only evidence of what went wrong is a timestamp that stopped.


Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

№ 5431