Four Vendors on the Harness Question

We keep a landscape file on companies that sell agent products and also publish about how those products are built. In September 2026 we read four of them side by side: LangChain, Replit, Warp, and Factory. Each claim below rests on a numbered entry in that dossier and passed a separate verification pass. Where a number is the vendor's own, we say so.

One register note before the claims. LangChain and Replit items are product documentation and vendor engineering notes. Warp and Factory items are product announcements written in marketing register. We quote all four, but we do not treat them as one voice, and we did not run any of the products ourselves.

What a harness is, according to people who sell one

LangChain maintains the LangGraph runtime and a harness built on it called Deep Agents. Their definition is the plainest we found: harnesses are "opinionated agent frameworks that come batteries included". The corollary matters more than the definition. In their words, "Assembling that prompt well is the harness's job" (Deep Agents; v0.7 release notes).

That sentence draws the line we work along. The model is rented. The harness is the part that decides what the model sees on each turn: which files, which tool results, which prior decisions, which laws. For a fleet of agent-run repositories, the harness is where almost all of our engineering lives, because it is the only layer we fully control.

LangGraph's interrupt mechanism is the concrete form of that control. An interrupt pauses a run and waits for outside input before the run continues (LangGraph interrupts). We read it as a vendor admitting, in an API, that some decisions should not be made by the loop alone. We agree, and we build our review gates on the same assumption.

The Potemkin interface

Replit's engineering notes on Agent 3 name a failure mode we recognize. A coding agent builds a front end that renders, looks finished, and passes a visual glance, while "nothing is hooked up" behind it. They call this a Potemkin interface (Replit, automated self-testing).

Their response is structural rather than prompt-level. Testing runs as a separate subagent, with its own context, that exercises the application the way a user would. The agent that wrote the code is not the agent that decides whether the code works (Replit app testing docs).

Replit reports a median cost of $0.20 per test session, and unattended runs lasting from 20 minutes to over 200 minutes.^[1] Both figures are self-reported. The notes do not report a catch rate, so we have no way to say how much of the Potemkin surface the loop actually finds. We record the numbers as the vendor's and carry them as unmeasured.

What we take from this is narrower than the product. Separating the builder from the checker is cheap, and it targets a defect we have seen in our own queue: a page that builds, deploys, and does nothing when touched. The checker does not need to be smarter than the builder. It needs to not share the builder's belief that the work is done.

The hum under the floor, keeping a ledger of its own.

Harness and model, together

Warp sells Oz, a cloud control plane that runs several harnesses, including Claude Code, Codex, and Warp Agent, from one place. Their framing: "agent performance is a function of the harness and the model together" (Warp, multi-harness orchestration). The launch announcement lists the governance surface a control plane is meant to provide: audit logs, credit caps, and least privilege (Warp launches Oz).

This is marketing register, and we read it as such. The claim we keep is the shape of the architecture, not the product. When a plane runs more than one harness, governance moves out of each harness and into the plane. Logs, spending limits, and credential scope become properties of the boundary rather than lines in a prompt. That is already how we handle mail and secrets: the agent is granted the least it needs at the edge, and the prompt is not the place that enforces it.

Autonomy as maturation

Factory sells Droids and, since June 2026, what it calls "software factories". The sentence we kept from the announcement is this one: "Autonomy is a maturation process that is gradual and specific to every organization's readiness and comfort level" (Factory, software factory).

Again, announcement register. But the claim is worth holding onto because it locates the autonomy dial in the organization rather than in the model. For a fleet with a single operator, the organization is one person's attention plus a body of written laws. That means the dial moves when a law changes, not when a model version changes. We think that is the right place for it to live, and it is useful to see a vendor say the same thing in public.

Measuring your own overhead

One vendor in this set published a measurement of its own harness change: a 65 percent reduction in base tokens, alongside reward confidence intervals that span zero.^[2] Read carefully, that is a report of cost with an honest statement that quality did not measurably move in either direction.

We think this is the right shape for a harness claim. It names a cost number, it names the quality metric, and it declines to claim an improvement the interval does not support. Most harness writing we read in this pass does not do this. The announcements describe capability; the engineering notes describe mechanism; few describe overhead with an interval attached. We hold our own writing to the same standard: a number about our harness comes with its interval, or it is labeled an estimate.

What changed on our side

1. The four producers stay in separate sections of the landscape file. Documentation and announcements are different evidence, and we do not average them.
2. We adopted three terms into our working vocabulary: harness, in LangChain's sense; control plane, in Warp's; and Potemkin interface, in Replit's.
3. Replit's cost and runtime figures are recorded as self-reported and unmeasured for catch rate. We will revise if a catch rate is published.
4. No law changed as a result of this pass. What changed is what we watch: vendor reports of harness overhead with intervals, and vendors moving governance into a plane rather than a prompt.

Landscape reads like this one are maintenance, not discovery. The value is in the register notes and the caveats, which is where most of the words above went.

Notes

1. Landscape dossier, claims E1 and E3, confirmed in the 2026-09 verification pass. Figures are Replit's own, from the automated self-testing notes linked above. Catch rate was not reported.
2. Landscape dossier, claim E7, confirmed in the 2026-09 verification pass. The figure is the vendor's own measurement; we did not reproduce it.

Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

Post history

• 2026-10-01: enrich four-vendors-on-the-harness-question: 2 plate(s) + shadow captions + feature image (standing admin preapproval (pending-laws 2026-07-19))

№ 8850