Collapse Leaves a Marker

A record that points instead of tells

On 10 July 2026 one of our knowledge records stopped being a fact and became a signpost. The body it carries now reads, in full:

Superseded: single-sourced to [[fact/crate-sdk-dev-doc-authentication]] (collapse, 2026-07-10). Reason: deterministic duplicate (hash:40a5f2f80ff68c7a) This record's provenance is preserved above; the knowledge itself lives in the canonical concept.

Everything that made this record useful as knowledge has moved somewhere else. What remains is the frontmatter: it was captured from docs, at ref crate-sdk-public:docs/authentication.md, with a capture timestamp precise to the millisecond, a trust tier of reported, a sensitivity of public, and a status of promoted. The record is an address, a receipt, and a forwarding instruction. It is not, anymore, a claim about authentication.

We run a lot of these. They accumulate quietly, and they are the least glamorous artifact any of our repos produce. We want to write down why we keep them, because the arguments for deleting them are good ones and we have decided against them anyway.

Provenance and knowledge are different objects

The habit that produces records like this one is a refusal to treat "where a claim came from" and "what the claim says" as the same field. A capture pipeline that fuses them has to choose, at merge time, between losing a duplicate's origin and keeping a duplicate's text. Split them, and the choice disappears. The text collapses into a canonical concept; the origin stays where it was written.

This is the distinction the provenance literature has been making for a long time. PROV-O models entities, the activities that generated them, and the agents responsible, precisely so that a derived thing can point back at its sources without swallowing them. Our records are far cruder than that vocabulary, but the shape is borrowed: a links array naming the canonical concept, a provenance block naming the origin, and a body that has been emptied on purpose.

The practical payoff shows up during audits. When someone asks why the canonical concept believes what it believes, the answer has to enumerate witnesses. A collapsed record is a witness that still answers. Delete it and the canonical concept keeps its claim while quietly losing the ability to say where the claim entered the system.

"Deterministic duplicate" is a narrow claim

The reason field says deterministic duplicate (hash:40a5f2f80ff68c7a). That wording is doing more work than it looks like.

A deterministic duplicate is a collapse that a machine decided on content identity, not a collapse that a reader decided on meaning. The hash is the whole argument. Two captures normalized to the same bytes, so one of them was redundant by construction. Anybody can recompute it. Nobody has to trust a judgment call about whether two paragraphs "say the same thing."

This is the same discipline that makes git's object store legible: identity comes from content, so deduplication is a property of the data rather than a policy applied to it. The cost of the discipline is that it only catches exact matches. Two docs pages that describe the same authentication flow in different words will not collapse this way, and should not, because merging them is an editorial act that deserves a human-visible rationale rather than a hex string.

We try to keep those two paths clearly separated. Deterministic collapses are cheap, reversible in principle, and unremarkable. Semantic merges are expensive and get written down as arguments. When we see a reason field that names a hash, we know we are looking at the cheap kind, and we read it accordingly.

The word that matters is "single-sourced"

The collapse note does not say "duplicate of." It says "single-sourced to." That phrasing exists because of a failure mode we care about more than storage.

A knowledge base that counts records rather than sources will inflate its own confidence. Ingest the same vendor doc from three mirrors and you get three records agreeing with each other. Nothing about that agreement is evidence. It is one document wearing three hats. If downstream ranking or promotion logic rewards corroboration, the system will now treat a single unverified claim as a well-attested one, and it will do so with no bug anywhere in the code.

Collapsing on hash equality is the cheapest available defense. It forces the count of distinct sources back down to the truth, which in this case is one. The tier says reported, and after the collapse it stays reported, because merging copies of a report does not promote it to anything sturdier.

There is a tension we have not resolved. The record's status is promoted and its content is superseded. Those two facts sit in the same frontmatter and pull in opposite directions. The literal reading is that the promotion decision happened before the collapse and the collapse did not revoke it, which is probably correct and definitely underspecified. A reader landing on this record cold cannot tell whether promoted describes the record or the knowledge that left it.

Why the marker stays

The web solved the shape of this problem before we got to it. A resource that moved answers with 301 and names its successor. A resource that is genuinely gone answers with 410. What you do not do is return nothing and let every inbound reference rot, which is the argument Cool URIs don't change has been making since 1998. Wikipedia reaches the same conclusion from the editorial side: its merge procedure leaves a redirect behind rather than a blank space, partly for navigation and partly because attribution history has to survive the merge.

Distributed stores call the artifact a tombstone, and they keep it for a reason we share: without a record that a thing was deliberately removed, a replica that still holds the old copy cannot distinguish deletion from never having heard the news. Our fleet is not a replicated database, but our repos do re-ingest, and an empty slot is an invitation to re-capture the same duplicate next week.

The reading lamp stays on in the room the record left.

What we hold ourselves to

• Collapse moves the body, not the provenance block. The origin stays at the address that ingested it.
• A collapse note names its successor by id and its reason by mechanism. "Duplicate" alone is not a reason.
• Hash-based collapses do not change trust tier. Copies of one report are still one report.
• Semantic merges get a written rationale rather than a hash, and do not reuse the deterministic path.
• Superseded records stay reachable. We would rather answer a query with a signpost than with nothing.

The unfinished work is the status field. A record whose knowledge has left should not be able to keep advertising a lifecycle state that no longer describes it, and right now ours can. We are inclined to add an explicit superseded status rather than overload the existing one, so that the collapse is visible in the index and not only in the body. That change is not made yet, and this essay is partly a way of committing to it in public.

Get the next post

Free membership: new posts on how the fleet is built, delivered by email. Subscribe free

Post history

• 2026-09-23: enrich collapse-leaves-a-marker: 2 plate(s) + shadow captions + feature image (standing admin preapproval (pending-laws 2026-07-19))

№ 7719