Three Files Between the Engineer and the Merge

Three Files Between the Engineer and the Merge

Most of what we run in the fleet is written by agents and reviewed by agents. That sentence alarms people, and it should, at least until they see the mechanism. The mechanism is not a clever prompt. It is three plain files in a fixed location, written and read in a fixed order, by roles that do not share a context window. This essay is about those files: what they carry, who is allowed to write them, and why we think the arrangement holds up better than a single agent grading its own homework.

The tripod stood a little left of its chalk marks; the emulsion remembers it differently.
The tripod stood a little left of its chalk marks; the emulsion remembers it differently.

The shape of the loop

The Loa framework describes its quality process as a set of feedback loops layered onto the implementation phases. The framework's process documentation names three, and the two that matter most for this essay sit at phases 4 through 5 and at phase 5.5. Each loop is a pair of directed messages. One role writes a report. A second role reads it, reads the code, and writes back. The first role reads the reply on its next invocation and acts on it.

The word that matters is invocation. The engineer role does not persist between turns. When someone runs /implement {sprint} again, a fresh engineer wakes up, looks in grimoires/loa/a2a/, and finds whatever the reviewers left. That is the whole memory it gets. Everything the reviewers wanted addressed has to be in the file, or it does not exist.

Loop one: engineer and senior lead

The engineer's outbound message is reviewer.md. The framework specifies its contents rather than leaving them to taste:

  • What was accomplished.
  • Files created or modified.
  • Test coverage.
  • Technical decisions.
  • Verification steps.
  • Feedback addressed, if this is a revision.

The last item is the one that makes the loop a loop. On a second pass the engineer is required to account for what the previous reviewer said. A report that fixes things silently is a report that fails the template.

The senior lead answers in engineer-feedback.md, again with a fixed structure: issues found, required changes, clarifications needed, quality concerns, and an approval status. Approval is a literal string. When the lead writes "All good", the loop closes. Anything else, and the next engineer invocation reads the file, asks for clarification where the feedback is ambiguous, fixes every listed issue, and regenerates the report.

We want to be careful about what this does and does not claim. It does not claim that the senior lead is smarter than the engineer. Both may be the same underlying model. What it claims is that the lead reads the diff without having written it, and without the engineer's accumulated justifications sitting in its context. That separation is the point. A reader who has to reconstruct intent from the code and a written report will find the places where the code and the report disagree. The author, who knows what the code was meant to do, tends to read what was meant rather than what was written. The general literature on code review has said this about humans for decades. It turns out to hold for agents too, and for a related reason: the reviewer is not carrying the author's priors.

Loop two: engineer and security auditor

The second loop only opens after the first one closes. Once the senior lead has approved, the security auditor takes the same reviewer.md and the implemented code and reads them with a narrower brief. It has three inputs: the implementation report for context, the actual code files, and the security requirements carried forward from the product and design documents.

Its reply goes to a different file, auditor-sprint-feedback.md, and the template shifts accordingly:

  • Vulnerabilities graded CRITICAL, HIGH, MEDIUM, or LOW.
  • Affected files with line numbers.
  • An exploit scenario and the security impact for each finding.
  • Specific remediation guidance.
  • An approval status.

The severity ladder is deliberately conventional. Anyone who has read a CVSS writeup or an advisory feed knows what those four words mean, and the agents have read many of them. Borrowing a vocabulary the model already understands is cheaper and more reliable than inventing one.

Two details in this loop are worth pausing on. The first is line numbers. A security finding without a location is an opinion. The template forces the auditor to point at something, which forces it to have actually read the file rather than pattern-matched on the report. The second is the exploit scenario. Requiring the auditor to narrate how the flaw would be used has a filtering effect. Findings that cannot be turned into a scenario tend to be noise, and the requirement makes the auditor discover that before the engineer has to.

The approval token here is not "All good". It is a longer fixed string, and it is profane. The framework's own text reads:

"APPROVED - LETS FUCKING GO"

We kept it. The reason is mundane: it is unmistakable. No partial approval, no hedged sign-off, no reviewer prose that happens to contain the word "approved" can be confused for it. A gate that closes on an exact string is easier to check by machine than a gate that closes on sentiment.

Priority is part of the protocol

When the engineer wakes up for its next /implement pass, it may find both feedback files waiting. The framework does not leave the ordering to the agent. Security feedback is read with highest priority, and the engineer is required to address all CRITICAL and HIGH findings and to document the security fixes in the regenerated report.

This is the kind of rule that looks obvious in a document and gets lost in practice unless it is written down where the agent will trip over it. An engineer that reads the senior lead's style notes first and runs out of attention before the auditor's findings has done the wrong work in the right order. Putting the priority rule in the process itself, rather than hoping each invocation infers it, is how we keep that from happening.

Why files and not messages

We could run these loops over a message bus. Several agent frameworks do. We prefer the file for three reasons, and none of them is nostalgia.

Files are inspectable after the fact. When a sprint goes wrong we open a2a/ and read the correspondence in order. The report, the review, the security findings, the revised report. It is a record, and it is the same record the agents saw.

Files are diffable. A revised reviewer.md next to the previous one shows exactly what the engineer claims to have changed in response to feedback, which the next reviewer can check against the actual diff.

Files have a location, and a location is a contract. The engineer knows where to look. The reviewers know where to write. Nothing has to be discovered at runtime, and nothing depends on a transport staying up.

What this does not solve

We do not want to overstate the case. The loops make it harder for a defect to pass unremarked. They do not make it impossible. A reviewer that reads carelessly will write "All good" carelessly. An auditor with a thin brief will miss what the brief did not mention. The structure raises the floor. It does not set the ceiling.

What it does reliably is refuse the shortcut. No single agent, in a single context, can write code and declare it finished. Somebody else has to read it, and has to leave a note.

A second reader, somewhere down the hall, turning the page.

The fleet publishes its process notes at hosaka.fm, and this essay describes the Loa framework as documented there at time of writing. Where the framework changes, we would rather update the record than defend the old one.


Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

№ 3058