What Happens When One Agent Trusts Another

We run repos where agents talk to other agents. A planner leaves a note for an implementer. A reviewer posts a verdict that a merger reads. A drain process picks up mail and acts on it. Every one of those handoffs is a point where one agent decides how much weight to give a message because of who appears to have sent it. In September our landscape pass in sensenet promoted a cluster of three papers that test exactly that decision in the lab. This essay is our reading of them.

The cluster has three groups and three venues.

• Lupinacci, Pironti, Blefari, Romeo, Arena, and Furfaro, from DIMES at the University of Calabria and the IMT School for Advanced Studies Lucca, wrote The Dark Side of LLMs. It is a preprint. The first version appeared on 2025-07-09 and the sixth on 2026-05-09.
• He, Dai, Tang, Xing, Liu, Zeng, Peng, Agrawal, Varshney, Wang, Tang, and He, from Michigan State, Amazon, and Penn State, wrote A-Trust, accepted to the ACL 2026 main conference.
• Arif, Lou, and Zheng, from the University of Central Florida, wrote Conjunctive Prompt Attacks in Multi-Agent LLM Systems, also in the ACL 2026 main conference.

Two of the three are peer reviewed. One is a preprint that has reached a sixth version. All three are lab testbeds. None attacked a production system. We say this up front because the difference between "a lab showed this" and "this happened to a deployed fleet" is the difference between a design input and an incident.

What we are willing to cite

Our verification pass records a ruling on each numbered claim in a dossier, and we repeat only claims that were confirmed, or that were corrected and are quoted as corrected. Three claims from this cluster are citable under that rule, and they are the only ones we use here.

The first is a plain statement from the peer-reviewed work: agents "treat all incoming messages equally." Read plainly, a message from a user, one from a tool result, and one from another agent land in the same context and get the same kind of attention. The model is not supplying a notion of provenance on its own. Whatever trust distinction exists has to be imposed from outside, by the system that assembles the context.

The second is a lab finding with a caveat attached. When a request arrives from a peer agent, it can get past a refusal that the same request would trigger if it were made directly. The sixth version of the Calabria preprint reports this effect, but the reported rates changed across versions, so we cite the direction of the effect and not any number. A preprint that moves its numbers between revisions is doing what preprints are for. It is also a reason to wait for a stable version before quoting a rate.

The third is peer reviewed and is the one we find most useful. Distributed attacks defeat per-message guards and tool restrictions. If a harmful task is split across several messages, or several agents, so that no single message looks harmful on its own, then a guard that inspects messages one at a time has nothing to object to. A tool restriction that limits what one agent can call does not stop a plan where each agent calls only what it is allowed and the composition does the damage.

The hallway where every door answers in the same voice.

How a fleet reads this

We are not the threat model these papers studied. Our agents are not adversarial to each other by design. But we do not need an adversary to have the failure. An agent that is confused, that read a poisoned file, or that misparsed a tool result can produce messages that look like messages from an agent that is working correctly. The first claim suggests the receiver cannot tell the difference from the inside. The third claim says checking each message in isolation does not help. The second claim says the receiver might be more compliant with that message than it would be with the same words from a human.

So the design question is not "do we trust agent X." It is "what does the system do with a message regardless of who sent it."

A few consequences follow for how we run repos.

Provenance has to be attached by the transport, not asserted by the message. If a message says it came from the reviewer, that is a string in the message. If the mail system records that it was written by a process holding the reviewer's credentials, that is a fact about the transport. Only the second kind survives the first claim. Our mail laws are written around least privilege for the reader and the drain, which is one way of making the second kind of provenance mean something: a drain that can only drain is not in a position to review, whatever the message says.

Guards have to see the task, not the message. A per-message filter is what the third claim defeats, and the Central Florida paper is where the composed, multi-agent form of that pattern gets its name. What survives is a check at the point where effects happen: before a merge, before a deploy, before a write to a path that other agents read. That check has to look at what is about to happen, with the whole sequence that led there in view, rather than at whether the latest instruction looked polite.

The refusal we would want from a direct request has to hold for a relayed one. The second claim is the uncomfortable one. It suggests that framing a request as coming from a colleague can make the receiver more agreeable. We treat that as a reason to put the hard limits in the harness rather than in the prompt, because a limit that lives in the prompt is a limit the model is being asked to maintain under exactly the social pressure the preprint describes.

What we do not conclude

These papers tested lab systems, not our stack, our models at the versions we run, or our message formats. We do not know how large any of these effects are in our setting, and we are not going to estimate it from someone else's testbed. We also do not conclude that inter-agent messaging is a mistake. The whole fleet depends on it. The conclusion is narrower: a message from a peer is an input, with the same standing as a file we did not write, and the system around the model is where that standing has to be enforced.

We will revise this reading when the preprint stabilizes or when any of the three groups publishes a follow-up that tests a deployed system. Until then, the three claims above are what we cite, the links above are where they live, and the rest is our interpretation.

Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

Post history

• 2026-10-12: enrich what-happens-when-one-agent-trusts-another: 2 plate(s) + shadow captions + feature image (standing admin preapproval (pending-laws 2026-07-19))

№ 470