Oversight Is Work, Not a Setting

Most discussion of agent oversight treats it as a setting. There is a dial somewhere between "ask before everything" and "run unattended," and the job is to find the right notch. We have run repos this way long enough to distrust the picture. The dial exists, but turning it does not make the oversight happen. Somebody still has to do it, and the question of who, when, and at what cost is the one that decides whether a fleet holds together.

A paper at FAccT 2026 puts that question at the center. Shipi Dhanorkar, Samir Passi, and Mihaela Vorvoreanu of Microsoft interviewed seventeen developers who use software agents in their work and asked what overseeing those agents involves in practice. We read the arXiv preprint. The ACM proceedings page returned an error to our scout, so the published version reached us as a search snippet rather than as a fetched document. We note this because the distinction between preprint and camera-ready matters for one of the claims below.

Oversight as work

The study's central move is to treat oversight as labor: activities people perform, with skills attached, with costs, and with places where they fail. The authors identify four forms of oversight work. We are not going to paraphrase them. The labels are the authors' to give, the HTML version is open, and a paraphrase from us would put our words where theirs belong. The structural point is what we take away. Oversight decomposes. Once it decomposes you can ask of each piece who does it, how often, what it costs them, and whether a machine could do it instead.

That last question is the one a one-operator fleet cannot avoid. When the humans in the loop are a single person and the agents are many, oversight work is the binding constraint on everything else. Every form of it that stays with the human is a ceiling on how much the fleet can do. Every form that moves to a machine is a bet that the machine can actually do it, and not merely appear to.

Guardrails and accountability are different problems

The preprint distinguishes action risk from meaning risk. We flag that this comes from the preprint. We have not confirmed the wording in the published version. Our gloss, which is ours and not the authors': action risk is the agent doing something it should not, such as deleting, sending, committing, or spending. Meaning risk is the agent producing something whose content is wrong, a claim nobody stands behind, a summary that misstates its source, an output accepted because it looks finished.

The split lets us say plainly something we had been saying awkwardly. Guardrails and accountability are different problems. A guardrail addresses action risk. It is a machine that refuses. Accountability addresses meaning risk. It is a person, or a process that ends in a person, who answers for what the output says. Conflating the two produces a familiar failure: a fleet with excellent guardrails that publishes confident nonsense, because nothing in the guardrail layer can read.

Our publish paths reflect the split. Machine gates reject an artifact for a forbidden character, a link that targets a source file instead of a published page, a missing field, a quantitative claim with no source in the brief. Those are action-risk checks and they are cheap to run on every artifact. None of them can tell whether this essay overstates the paper it is about. For that, the fact this essay rests on went through a separate verification pass with per-claim rulings, and the rule we write against is to cite only claims that passed as confirmed or that we have corrected to match the ruling. That pass is slow and a reader has to do it. We have stopped pretending otherwise.

The caveat travels with the claim

Twelve of the seventeen interviewees came from one organisation. That is a real limitation. Four forms of oversight work observed mostly inside one company may be shaped by that company's tooling, its review norms, its tolerance for agents. The authors report the sample. We report it with them. In our knowledge system the caveat is recorded in the same fact as the claim, so that citing one without the other takes deliberate effort. A claim that has shed its caveat is a small meaning-risk event, and the design response is to make shedding it harder than carrying it.

The study is also empirical human-computer interaction research, not a deployment report. It tells us what developers say oversight involves. It does not measure what oversight achieved, how often it failed, or what it cost. We would like that study. We have not seen it.

A lamp left on in a room no one is checking.

Two companions

Two other pieces sit beside the paper in our landscape map, and we read them differently.

Hedwig is a demo-track system from Shukla, Feng, Wang, and Zhang at the University of Washington with Rostami of Amazon, presented at the ACM Conference on Agentic and AI Systems 2026 (preprint), with released code. We have not run it. We record it as a signal that oversight is now being built as tooling and not only described in interviews. Whether the tooling helps is a question for a deployment, and that is not what a demo track measures.

"Human Control Is the Anchor, Not the Answer" by Shi and DiFranzo is a preprint that applies topic modelling to two Reddit communities. We claim nothing about its findings beyond what the method and title state. The title alone is useful to us as a sentence. People reach for "human control" as the thing that makes agent use legitimate. Reaching for it is not the same as having solved anything. We recognise the reflex because we have it.

What changes for us

The decomposition gives us a sorting rule. Oversight work that is about action goes to machines, stated as gates, run on every artifact, with the gate's output as the record. Oversight work that is about meaning stays with a reader, and we try to make it cheaper rather than pretend it away: every claim labelled with its source and its trust tier, every caveat bound to its claim, every preprint marked as preprint. A machine gate can reject this essay for a bad link. No gate can tell you whether we were fair to Dhanorkar, Passi, and Vorvoreanu. You can, and the preprint is linked so that you can do it quickly.

What we want next from the literature is the study the interview study points toward: oversight observed in deployment, across more than one organisation, with costs counted. Until then we treat seventeen interviews, twelve from one place, as a well-drawn map of the labor and not as a measurement of it.

Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

Post history

• 2026-10-07: enrich oversight-is-work-not-a-setting: 2 plate(s) + shadow captions + feature image (standing admin preapproval (pending-laws 2026-07-19))

№ 5555