Keying design
How content keys, container keys, and recipients fit together.
Published from docs/keying-design.md (GitHub)
This document describes the access and key-delivery model: signed access manifests, the container KEK hierarchy that delivers document and blob keys, local-first container creation, and how the protocol treats unauthorized grants, rollback, split views, and first-contact identity keys (see security-guarantees.md).
HTTP routes use the signed mutation surface summarized in api-architecture.md.
For shared protocol terminology, see glossary.md.
The signed access-event, access-manifest, container KEK, document content-key, blob content-key, principal-policy, write-header, and local checkpoint primitives define the executable keying boundary. Transparency deployment, first-contact identity trust, verifier use at every client encryption decision, and historical replay are separate deployment boundaries.
Design Properties
- Additive sharing avoids per-document and per-blob recipient fanout.
- Encryption targets are derived from verified signed access state, not from server-provided recipient lists.
- Access and key state are verified through shared protocol code rather than UI convention.
- Server-visible structure remains the input to access derivation.
- End-to-end encryption is preserved: the server stores ciphertext and wrapped keys, not plaintext content keys.
- Revocation is forward-looking unless content is re-encrypted.
- Local-first creation uses signed local intents rather than fake server access hashes.
Boundaries
- The protocol does not retroactively revoke bytes or keys already distributed.
- The protocol does not hide container, document, or blob structure from the server.
- The protocol does not prevent denial of service, withholding, or delayed publication.
- First-contact identity trust requires a trust root outside mutable API rows.
- Public blockchain anchoring is optional and is not the primary authorization system.
Core Design
Keying has four distinct planes:
- Identity and principal policy.
- Signed access manifests.
- Container KEK key delivery.
- Content-object encryption for documents and blobs.
The API can store and project these planes, but an honest client must be able to verify the authority for each plane independently.
Threat Model
Assume:
- client code is honest and runs the verifier;
- user/device signing keys and private encapsulation keys are uncompromised;
- cryptographic primitives hold;
- the API, database, object storage, cache, and read replicas may be malicious, stale, or inconsistent.
The server may still deny service, omit data, or delay publication. The keying protocol is designed so those attacks become availability failures, not silent confidentiality failures.
Identity And Principal Policy
Principal policy uses the signed group/organization model with two rules.
First, any principal transition that removes a reader, demotes a role that can read, disables a member, or changes a member key in a way that shrinks future access must advance the principal key epoch. Clients and the API both reject shrinking policy transitions that reuse the old principal key epoch. Removing a container grant must advance it too, since wraps sealed to the old key are retained; both enforce that.
Second, user identity keys need an authority outside mutable API rows, such as:
- out-of-band key verification or invitation links that bind
userIdto an identity signing key fingerprint; - organization-admin signed invitations that name the recipient identity key;
- an append-only identity transparency log with client checkpoint pinning;
- a combination of the above.
Production clients implement the first option as identity links: a link binds the user ID, trust domain, algorithms, and both public keys. Every admission (contact, roster user, group member, share recipient, roster profile) requires a pin that a link or the device’s own keys established; pin provenance keeps a first-sight pin from standing in. Signer resolution and other verification of referenced users still pin on first sight, so first-contact key substitution remains possible for an identity no link has verified. No object keying algorithm can fully fix that by itself.
Organization administration uses reserved groups: Admins authorizes admins,
and Members drives the roster. Admins is not nested into Members; instead
a managed principal may name only active roster entries. The signed organization
payload binds both IDs, and Admins contains only admin users. Each signed
principal state uses a null externalAuthority for direct authorization or the
exact Admins head for external authorization. Citations match exact signed
Admins states without in-chain rollback; only API commits require the current
head. Display/read-model projections never provide keying inputs.
Signed Access Manifests
Every security-relevant graph mutation is represented by a signed access event and a derived access manifest.
Protected manifest scopes:
- organization;
- container;
- document link set;
- blob attachment/target set, derived from signed attachment binding events.
The API may store materialized projection rows, but projection rows are not authority. A client verifies manifests and derives projections locally.
Access Event Shape
Each mutation signs canonical bytes for an event like:
type AccessEvent = {
version: 1;
eventId: string;
eventType:
| "container.create"
| "container.grant"
| "container.revoke"
| "container.rekey"
| "container.recite"
| "container.move"
| "document.link"
| "document.unlink"
| "document.purge"
| "attachment.bind"
| "attachment.detach";
objectKind: "container" | "document" | "blob";
objectId: string;
organizationId: string;
previousManifestHash: string | null;
dependencyManifestHashes: string[];
bodyHash: string;
signerUserId: string;
signerDeviceId: string;
signerKeyFingerprint: string;
signedAt: string;
signature: string;
};
The event body is type-specific. For example, container.grant includes the
subject principal, access level, current referenced principal policy head, and
the container manifest hash being advanced. document.link includes the
document manifest hash and target container manifest hash. attachment.bind
includes document id, slot id, blob id, expected binding id, and document
manifest hash. Attachment events use objectKind: "blob" when they advance
the blob target/binding manifest; the attachment-specific identity lives in the
event body, not in a separate fourth manifest kind.
Manifest Shape
An access manifest is a canonical snapshot derived from events:
type ReferencedPrincipalHead = {
principalType: "group" | "organization";
principalId: string;
version: number;
keyEpoch: number;
stateHash: string;
keyFingerprint: string;
};
type AccessManifest = {
version: 1;
objectKind: "container" | "document" | "blob";
objectId: string;
organizationId: string;
epoch: number;
previousManifestHash: string | null;
eventHash: string;
structuralHash: string;
grantRoot: string;
referencedPrincipalHeads: ReferencedPrincipalHead[];
keyTargetHash: string;
};
type AccessManifestHash = string;
AccessManifestHash is the digest of the canonical serialization of
AccessManifest. It is stored and referenced beside the manifest, not inside
the manifest payload being hashed.
Clients verify:
- event signatures;
- signer identity key authenticity;
- signer authorization against the previous manifest;
- referenced group/organization policy chains;
- cross-object dependencies, such as parent container, linked container, or active attachment binding heads;
- monotonic local checkpoint rules;
- optional transparency inclusion and consistency proofs.
This turns “the API returned a self-consistent grant view” into “the API returned a verifiable signed state transition.”
Container Access Manifests
A container access manifest is the signed head for the container’s direct access and structural parent edge. It does not contain an expanded recipient list for every descendant document.
Container manifest inputs:
- container id and organization id;
- parent container id and parent manifest hash;
- authoritative metadata document id for the container;
- direct grants on this container;
- referenced principal policy heads for direct managed-principal grants;
- current container KEK epoch id;
- previous container manifest hash;
- authorizing signed event hash.
The effective access for a container is derived by walking ancestor container manifest heads and unioning their direct grants. For runtime efficiency, the server may materialize ancestor paths and effective grant summaries, but a client can verify the path from signed manifest hashes.
Important scaling rule: sharing or revoking a container must not rewrite every descendant container manifest. Descendants inherit through their parent edge and their parent KEK wrap. A descendant manifest changes only when that descendant’s own direct grants, parent edge, or key epoch changes.
Document Link-Set Manifests
A document link-set manifest is the signed head for the set of containers that currently link to a document. It is not a per-container access manifest and it does not expand users.
Document link-set inputs:
- document id and organization id;
- active linked container ids;
- dependency container manifest hashes used to authorize the link/unlink event;
- previous document link-set manifest hash;
- authorizing signed event hash.
document.link and document.unlink events advance only this link-set head.
They do not copy container grants into the document and they do not mutate
container manifests.
The document’s write targets are derived at write time:
- verify the document link-set manifest;
- load and verify each active linked container’s current manifest head;
- resolve each linked container’s current container KEK epoch;
- sort and hash those KEK targets.
This means a container grant change does not cause writes to thousands of document link-set manifests. Existing documents see the new container KEK state when their next write target set is derived.
Derived Document Access Views
A document access view (VerifiedDocumentKekTargets) is a deterministic
projection:
type VerifiedDocumentKekTargets = {
documentId: string;
linkSetManifestHash: string;
linkedContainerManifestHashes: string[];
linkedContainerKeyEpochIds: string[];
targets: DocumentContentKeyTarget[];
documentKeyTargetHash: string;
};
This view can be cached, indexed, and returned by the API, but it is not the authority. Clients recompute and verify it from signed manifests before using it to encrypt.
The same pattern applies to blobs. Blob access is a derived view from signed active attachment binding events plus the linked documents’ verified target sets.
Authorization Rules
container.create: signer must have write access to the parent container.container.grant/container.revoke: signer must have admin access to the container.container.rekey: signer must have current write access to the container and may only advance the KEK epoch for an already-authorized manifest. It cannot change grants, parentage, or linked objects.container.move: signer must have admin access to the moved container and write access to the destination parent; the event references both manifest heads.document.link: signer must have write access to the document through at least one current linked container and write access to the target container.document.unlink: signer must have write access to the document through at least one remaining linked container and write or admin access to the unlinked container.attachment.bind/attachment.detach: signer must have write access to the document under the current document manifest.
These rules treat link as additive and unlink as potentially subtractive. A writer who can edit the document can already disclose its plaintext through ordinary content edits, so requiring write access through every linked container would add friction without preventing a meaningful confidentiality attack. Unlink remains stricter because it can remove another container’s future access path.
Container KEK Hierarchy
Each container has a current container KEK epoch.
Container KEKs are wrapped to:
- direct user identity/device encapsulation key epochs;
- group or organization principal key epochs;
- the parent container KEK epoch when inherited access should flow downward.
Documents and blobs do not wrap content keys directly to every effective user. They wrap content keys to current container KEK targets.
Why Parent KEK Edges Exist
If child containers always wrap their KEK to the parent container KEK, an additive share on an ancestor can be one new wrap of the ancestor KEK. The new recipient follows the existing KEK chain to descendant content. That is the main fanout win.
The tradeoff is explicit: compromising an ancestor KEK compromises the subtree that inherits from it. That matches inherited access semantics, but it must be treated as the blast-radius boundary.
Container Key Epoch Database Row
type ContainerKeyEpoch = {
id: string;
containerId: string;
keyEpoch: number;
accessManifestHash: string;
parentContainerKeyEpochId: string | null;
predecessorContainerKeyEpochId: string | null;
predecessorBridgeVersion: number | null;
predecessorBridgeSuite: string | null;
predecessorBridgeIv: string | null;
wrappedPredecessorKey: string | null;
keyringIv: string | null;
sealedKeyring: string | null;
createdByEventHash: string;
createdByManifestHash: string;
createdAt: string;
};
Every epoch after the first stores two immutable rotation artifacts, both committed by hash in the signed move, rekey, or revoke event so server-side ciphertext substitution is detectable before decryption.
The predecessor bridge is the append-only log entry: the immediately preceding KEK encrypted under the new KEK, binding the container id and both committed epoch ids as AES-GCM additional data. It is written exactly once by the rotator — the only party the protocol guarantees held both keys — and is never rewritten by any later rotation. The bridge’s wrapping suite and version are persisted per row so a suite change is representable.
The sealed keyring is the snapshot read path: the container’s complete
predecessor key history — one fixed-width record per epoch, in ascending
order — AEAD-sealed under the new KEK. A client that unwraps the current KEK
opens the keyring in one decrypt and holds every retained historical KEK; no
chain walk occurs on the hot path. Entry ordinal i is key epoch i + 1, so
the sealed ciphertext length for key epoch n is an equality
(8 + (n - 1) * 64 + 16 bytes), which the server enforces at write time on a
blob it cannot decrypt and clients enforce again after opening. Each recovered
entry is verified against its material-committing epoch id before use.
Initial epochs have every rotation-artifact field null; later epochs have all of them populated (a database check constraint enforces the all-or-none shape). The keyring is derived, rebuildable state; the bridge log is ground truth.
Container KEK epochs use ids of the form
tearleads.container-kek.v2.sha256:<hash>. The domain-separated hash commits to
version 2, the container id, numeric KEK epoch, and the canonical ML-KEM wrapping
public key deterministically derived from the 32-byte KEK. Servers and child-only
writers verify the published key against this commitment without an ancestor
secret. KEK holders derive that same public key from recovered material and check
the commitment independently. Only the v2 format is accepted.
The wrapping public-key cache is bounded to 128 entries and indexed by container id plus a process-local HMAC token of the key bytes. A second cache retains up to 65,536 public material-id commitments, keyed by container, numeric epoch, and the same token. Neither cache retains KEKs, private wrapping keys, or derivation seeds; temporary copies and token buffers are zeroized. The non-extractable HMAC key is generated once per process, so tokens cannot be correlated across runs.
Every epoch-creating child event has an explicit signed parent citation.
container.rekey and container.revoke include parentManifestHash in their
bodies, naming the current parent used to wrap the new epoch (null for roots).
The derived state retains that citation for historical recovery. Selection uses
that exact hash; withholding the cited head causes a missing-dependency refusal.
The crypto verifier also handles two cited heads of the same parent as defense
in depth; the API and SDK reject such duplicate-container dependency lists.
Container Key Wrap Row
type ContainerKeyWrap = {
containerKeyEpochId: string;
recipientKind: "user" | "group" | "container";
recipientId: string;
recipientKeyEpochId: string;
recipientKeyFingerprint: string;
kemCipherText: string;
wrappedKey: string;
wrapManifestHash: string;
};
The wrapManifestHash binds the wrap to the manifest that justified it. A
client rejects wraps whose recipient key epoch, principal policy head, parent
container key epoch, or manifest hash does not match the verified graph.
The client also rejects wraps that decrypt to different KEK material than the
signed epoch id commits to.
Additive Versus Subtractive Changes
Additive changes:
- advance the signed access manifest;
- keep the current container KEK epoch;
- add one or more wraps for the existing KEK;
- do not touch descendant document/blob content-key bundles.
Subtractive changes:
- advance the signed access manifest;
- create a new KEK epoch for the changed container;
- require a bridge from that new KEK to the immediate previous KEK;
- require a keyring holding every predecessor KEK sealed under the new KEK;
- require descendants whose future access depends on the changed ancestor KEK to move to a post-change KEK epoch before accepting future writes;
- do not rewrite old document/blob bytes unless a separate re-encryption job is requested.
Lazy descendant rekey is acceptable, but writes must fail closed. If a document write targets a container whose current key epoch is still reachable through a revoked ancestor key, a post-revocation descendant KEK epoch must be materialized before the write is accepted.
That materialization is a key-maintenance transition, not an access-control
transition. A current writer may submit a signed container.rekey event when
the event only advances the KEK epoch for the already-verified manifest and
wraps the new KEK to the exact current targets. Admin rights are still required
for grant, revoke, and move events. The API can coordinate the transaction and
validate targets, but it cannot synthesize wrapped key material because it does
not know the plaintext KEK. For committed KEK epoch ids, a client that unwraps
server-supplied replacement material rejects it unless the material hashes back
to the id signed in the manifest.
Current access is history-inclusive. A newly admitted user receives only a current recipient wrap, then opens the sealed keyring for every retained historical KEK; reading history needs no old user/principal envelope. Revocation remains forward-only because a user who possessed an old KEK may retain it, but a current user does not depend on a retained device, an old principal key, or a document rebaseline to read retained history. Fresh baselines remain a sync optimization and content-key rotation mechanism, not a key-recovery or liveness dependency.
This also exposes retained-history metadata to every current member even before decryption: keyring size (and therefore rotation count), material-committing epoch ids, access-manifest hashes, and parent epoch references. The design accepts that metadata disclosure as part of history-inclusive access; only the old plaintext KEKs and content remain sealed.
Reads and recovery projections may contain descendants pinned to historical parent KEK epochs. The verifier reconstructs that epoch’s binding from the parent’s complete verified signed manifest lineage, including its creation event’s parent citation. Missing history, unrelated epochs, and forged bindings fail verification. The projection needs no additional historical epoch records. The current parent’s sealed keyring supplies the historical key needed to open the child. Future writes still require current parent epochs throughout the path: lazy descendant rekey must complete before accepting new content. SDK write planners enforce this before encrypting or wrapping new material, even if the API supplies a valid recovery projection. The API checks the full parent path before accepting new container epochs; parent-first repairs remain valid.
History delivery is one round trip and one decrypt regardless of rotation count, whereas walking the bridge log needs one dependent unwrap per epoch. The work is still linear: the sealed blob is 64 bytes per retained epoch, so transfer, AEAD processing, and per-entry material-id verification all grow with epoch count. The difference is shape — one bulk decrypt plus independent entry checks in batches of 16 that yield to the event loop, rather than a chain where each unwrap gates the next. At the epoch cap the sealed keyring is about 4 MB, which is why the kek-log serves at most one historical keyring per request. Rotation pays the linear cost instead — the rotator opens the previous keyring and re-seals it plus the retiring key under the new KEK — which is one decrypt, one seal, and 64 bytes per retained epoch.
The v2 material check derives an ML-KEM public key on a cache miss. The compact commitment cache covers one full history at the protocol epoch cap; interleaved histories larger than the cache evict entries. Cold work is linear and can take tens of seconds near the 65,536-epoch cap. Bounded batches prevent an unbounded burst of pending derivations and keep the event loop responsive. Verification before re-sealing is intentional: skipping unused entries would let an honest rotation launder poisoned history into its new signed commitment.
There is deliberately no depth cap and no truncation: the keyring for epoch
n must contain exactly n - 1 entries, over- and under-length payloads are
rejected by the same equality, and a keyring that omits an epoch fails the
structural entry-count check. The MAX_CONTAINER_KEY_EPOCH write-time bound
(65536) is a runaway-rotation backstop sized to be unreachable by legitimate
use, enforced when a rotation is accepted — never against existing data — so
it can never make retained ciphertext unreadable. Every keying bound and what
lies past it is tabulated in limits.md.
Every keyring entry is checked twice on the read path: against the material-id its ordinal commits to, and — for any epoch the projection’s signed material names — against that committed epoch id. The second check is what stops a self-consistent forgery: a poisoned keyring can mint a fresh key with an id that commits to it, which the first check alone would accept. Rotations apply the same material check before re-sealing, so an honest rotator never launders history it did not verify.
A keyring that fails verification is a hard integrity failure for the affected
historical epochs, but it is cache poisoning, not data loss. The client
reports the failure when an operation needs one of those epochs, retains its
independently verified current KEK so corrupt history cannot brick
current-epoch reads, and can rebuild ground truth from the append-only bridge
log served by GET /containers/:id/kek-log: walking the write-once bridges
from the current KEK recovers every predecessor, each checked against its
material-committing epoch id. Repair is an ordinary container.rekey whose
keyring is sealed from the rebuilt entries; the poisoned artifact stays on
record, attributed to the signed event that committed it.
The bridge log has no repair story of its own — that is why it is never
rewritten. A bridge that fails to decrypt severs log-based recovery for the
epochs below it, but every epoch’s recipient wraps are retained forever
(cross-epoch wraps are never deleted; this is a stated protocol invariant,
not an implementation accident) and are served by the kek-log alongside the
bridges. A member holding a direct user envelope at epoch i can
therefore recover K_i from their identity keys plus server state, written by
the epoch-i rotator and untouchable by any later writer, and re-anchor a
repair rekey from there.
That backstop has one documented bound. A group- or organization-addressed
envelope additionally requires the principal secret key for the key epoch it was
addressed to, which resolves only through principal-policy bundles the client
can reach; after a principal key rotation a pristine client cannot reach the
older ones. The client fails closed with a distinct
HistoricalWrapUnavailableError (reason principal-key-unreachable) rather
than reporting corruption, so the condition is diagnosable. The API serves no
historical principal-key envelopes by design
(protocol-specification.md).
The only wholly unrecoverable case is a sole-ever member destroying their own
history.
formal/container-keying/KeyringReachability.tla
model checks this composition: the log alone recovers everything while bridges
are intact, severance damage is bounded to epochs below the broken link, an
honest current keyring implies full recoverability, and retained wraps backstop
a poisoned snapshot.
The server cannot prove that sealed rotation artifacts actually decrypt to the keys their ids commit to. An authorized writer that deliberately signs well-formed but undecryptable ciphertext can deny access to older history up to the limits above. The supported clients round-trip both artifacts while constructing mutations, but this is not an availability guarantee against a malicious writer. Database backup only repairs later storage damage; it cannot repair deliberately invalid ciphertext that was originally accepted.
A container move creates fresh successor KEK material as well as a new epoch identity because the parent binding changes. It carries the same two rotation artifacts as rekeys and revocations: the write-once bridge and the re-sealed keyring.
Document Content Keys
A document has one canonical content-key bundle per document key epoch.
Document write targets are the current KEK epochs of every linked container:
type DocumentContentKeyTarget = {
containerId: string;
containerManifestHash: string;
containerKeyEpochId: string;
containerKeyEpoch: number;
};
For a content write, the client:
- verifies the document manifest and linked container manifests;
- verifies all referenced principal policies and identity keys;
- derives the sorted target list;
- computes
documentKeyTargetHash; - verifies that the current canonical document content-key bundle commits to that exact target hash;
- signs the encrypted write header over object id, document epoch, manifest hash, target hash, update metadata, and ciphertext hash.
The API rejects writes when submitted targets do not exactly match current verified targets. Readers verify the signed write header before trusting target metadata.
Write authorization requires one linked write path. Reusing a current content-key bundle needs its document key, not every linked container’s KEK. Each write still commits to all targets so every linked reader observes the same document state.
This is not an independent-progress guarantee across rotations. Creating or
rotating the bundle requires wraps for every current target KEK, and the
symmetric content-key wrap needs those target keys. A writer with access to
only some links can therefore wait for a member able to repair the other
targets and materialize the complete bundle. Even a readable target can require
another member’s repair when its path is stale and the writer lacks write
access there. The SDK keeps the queued edit and reports
document_ancestor_repair_inaccessible when another verified linked path still
grants write access; losing all linked write access remains an authorization
refusal. A later sync can resume after repair.
This design retains the privacy boundary on inaccessible linked paths: it does not disclose their current grants to enable public-key-only repair. Public target wrapping or envelope contributions would need a separate authenticated protocol; the write-liveness claim assumes neither.
Content Write Authorization
A content write-header signature proves which device produced the ciphertext metadata, but it does not by itself prove that the device was allowed to write the object. Because any reader with the current content key can usually produce fresh ciphertext under that key, write verification must treat write authorization as part of the signed-header proof.
verifyWriteHeader must verify that:
- the signer identity/device key is trusted for the signed identity state referenced by the header;
- the header commits to the object id, organization id, object kind, content-key epoch, access manifest hash, key target hash, encryption suite, nonce or subkey derivation inputs, update metadata, and ciphertext hash;
- the committed access manifest and referenced principal policy heads verify;
- the signer had write access under the committed manifest state, not merely read access and not merely a mutable API-authenticated session;
- the committed key targets exactly match the verified graph for that manifest state.
Historical writes are verified against the manifest state they commit to, while new writes must also satisfy the latest-state and local-checkpoint rules before the API accepts them. For documents linked to multiple containers, the authorization rule is that write access through at least one active linked container authorizes a document content write, and that write is then encrypted to the full verified target set for all active linked containers.
AEAD Nonce And Subkey Discipline
Documents and blobs are multi-writer objects. Multiple devices may encrypt different ciphertext records under the same content key epoch, including while offline, so must not rely on local counters or implicit library defaults for nonce uniqueness.
Every encrypted content record needs a verifier-enforced encryption domain. The rule is:
- assign a globally unique, high-entropy
contentRecordIdfor each encrypted update, baseline, blob version, or replacement object, such as a UUIDv4 or another identifier with at least 128 bits of CSPRNG randomness; - derive a per-record AEAD key with domain-separated HKDF inputs that include
protocol version, organization id, object kind, object id, content-key epoch,
encryption suite, and
contentRecordId; - generate a fresh 96-bit AES-GCM IV for every encryption attempt and commit
it in the encrypted record bytes covered by the signed
ciphertextHash; - include the derivation-domain fields as AEAD additional authenticated data;
- commit those fields in the signed write header;
- reject duplicate
contentRecordIdor duplicate derived nonce domains for the same object/content-key epoch.
For document updates, the header’s signed metadataHash also commits a
domain-separated HMAC of the plaintext update bytes. Its key is derived by
HKDF from the document content key and record domain, so authorized readers can
verify it without exposing an offline plaintext-confirmation oracle to the
server. A content-key holder can also generate a commitment for arbitrary
plaintext, so this binds the record against server or storage substitution, not
against a malicious authorized reader. Readers authenticate the metadata,
decrypt the record, and reject a plaintext-hash mismatch before importing it
into Loro. Retained history thereby keeps a keyed identity for an authored
plaintext while retaining its ciphertext, provided the signed header,
original content-record id and epoch, and corresponding content key remain
available. A full-history rotation snapshot’s commitment is not comparable to
its constituent per-update commitments, so this is weaker than per-update
re-encryption with inner author signatures, which the protocol does not
implement.
The content-record suite is aes-256-gcm-hkdf-sha256-record-key. A content
writer supplies a UUIDv4 contentRecordId, commits the suite in the signed
write header, and commits a nonceDomainHash over the protocol version,
organization id, object kind, object id, content-key epoch, suite, and
contentRecordId. The content record domain is the input to per-record key
derivation, and the encrypted record carries a fresh random AES-GCM IV.
Storage enforces uniqueness for both contentRecordId and nonceDomainHash
within the same object/content-key epoch, while the random IV keeps accidental
same-domain re-encryption from reusing the same AEAD key/nonce pair before the
duplicate write reaches server storage. The invariant cannot vary: honest
concurrent writers must be unable to reuse the same AEAD key/nonce pair for two
different plaintexts.
For additive changes, the document content key may be reused and wrapped to a new target if the target set only grows. For shrink, subsequent writes require a new document content-key epoch and a fresh baseline. Revocation remains forward-only unless old ciphertext is re-encrypted.
Content-Key And KEK Wrapping Suites
Content-key and KEK wraps do not use the content-record HKDF suite. The wrap suite identifiers are:
- document content-key to container KEK:
tearleads.document.content-key-wrap.aes-256-gcm-container-kek - blob content-key to container KEK:
tearleads.blob.content-key-wrap.aes-256-gcm-container-kek - container KEK to user or managed-principal key:
tearleads.container-kek-wrap.ml-kem-1024-aes-256-gcm - container KEK to parent-container KEK:
tearleads.container-kek-wrap.ml-kem-1024-aes-256-gcm-parent-kek - successor container KEK to predecessor container KEK:
tearleads.container-kek-wrap.aes-256-gcm-predecessor-kek - predecessor keyring sealed under the current container KEK:
tearleads.container-kek-keyring.aes-256-gcm-current-kek
Document and blob content-key target envelopes carry wrappingMetadata.suite
and an AES-GCM IV; the AAD binds suite, object id, content-key epoch and
target. Container KEK wraps carry no suite field: recipientKind selects the
user/managed-principal ML-KEM wrap path or the parent-container ML-KEM wrap
path.
Every keyed container signs containerKeyPublicKey in its access event and
manifest key-target commitment. Grant and recitation events preserve it with
the KEK epoch. Creation, rekey, revoke, and move events bind the replacement
public key. The encoding is canonical base64, with the ML-KEM-1024 length and
coefficient checks from FIPS 203 section 7.2. Unkeyed states carry null for both
the KEK epoch and public key.
The parent wrapping key pair is derived from its 32-byte KEK using HKDF-SHA-256:
64 output bytes, UTF-8 salt
tearleads.container-kek.parent-wrapping.ml-kem-1024.v1, and UTF-8 container ID
as info. Those bytes seed ML-KEM-1024 key generation. Readers check the derived
public key against the signed current state; retained historical KEKs derive
the corresponding historical decapsulation keys. A child-only writer can wrap
a new child KEK to the verified parent public key without learning its secret.
The parent recipient fingerprint is the signed epoch-record hash. The child
epoch’s parent pin must equal the parent epoch named by its signed creation
event’s verified dependency citation, even when the pin names the current
parent. A self-consistent epoch record and wrap target cannot substitute
another parent epoch. A later grant or recitation cannot stand in for the first
signed manifest of that key epoch. Historical parent resolution is enabled only
for read/recovery; write verification still requires the current parent epoch.
All authors and readers use public parent wraps; there is no symmetric parent-wrap path. Each parent KEM ciphertext is 1568 bytes, and each signed public key is 1568 bytes before base64 encoding. Public keys add public metadata, not access to ancestor keys. Whole-path write-currency checks remain necessary: a stale intermediate key can be known to a revoked ancestor holder, so descendants must not wrap future keys to it. Public wrapping enables independent repair immediately below a current parent; it does not reach further up.
A writer whose only grant sits below a stale intermediate therefore cannot restore currency itself. A rekey is authorized over the root-to-target path, so a grant further down never counts, and the writer is never given the intermediate’s key: besides being reachable by the revoked holder, it opens the intermediate’s other children, which that grant does not cover.
No device may wait on another device’s write, so that state is never committed.
The invariant is that every proper ancestor of a directly granted container has
a current parent edge. A rekey, revoke, or move must carry, in its own
transaction, the rekeys of every descendant that sits above a directly granted
container; the API refuses it otherwise and names them. A self-revoker losing
write access on owed levels is refused with container_descendant_rekeys_inaccessible;
another authorized member must complete that revoke. Retained access inherits downward.
A container’s first direct grant requires the chain above it to be current, and
the sharer repairs a lazily stale path first; those repairs need not be atomic,
since a repair never strands anyone. A container that already carries a grant
owes nothing more for a further one. Inline repairs inside document and blob
writes are checked against the same rule, though under it a stale container is
never above a granted one, so an honest inline repair cannot trip it. With
those levels current and within the carried-rekey bound, a grantee repairs its
own subtree independently. A document linked outside that subtree can still
wait on another member’s repair, as described under content-key bundles above.
Containers with no grant beneath them still repair lazily, first writer wins,
which keeps a rotation’s cost proportional to what is shared below it rather
than to the subtree. A child container carries no direct grant unless shared, so
the carried set is usually empty. It is capped at 64 per rotation; past the cap
the remainder repairs lazily rather than refuse a revocation, and a writer who
meets such a level parks under an explicit
document_ancestor_repair_inaccessible state until the dependent-path hint that
follows its repair; limits.md
records that trade. Group rematerialization is held to the same rule: a
rekey or revoke among a policy’s containers carries what it owes in its batch,
parent-first under the rotation it rides; a level its rotations strand between
two of its containers is carried unnamed; and each entry cites the heads
earlier entries leave, grants included.
The inaccessible-intermediate repair model
covers the rule and the alternatives it rejects.
Blob Content Keys
Blob targets derive from the union of active signed attachment bindings.
For each active binding:
- verify the attachment event signature;
- verify the referenced document manifest;
- derive the document’s linked container KEK targets.
The blob content key is wrapped to the union of those container KEK targets. A bind submits only its new binding’s targets and target hash. The API holds the blob row lock, checks that exact scope, carries every other stored binding’s envelopes byte for byte (including retained historical epochs), and recomputes the merged stored hash. Read access to an existing blob never permits replacing another binding’s key material.
For fresh link-set attachment envelopes, the SDK indexes KEKs by the full container, manifest, epoch identity, and epoch number derived from verified current paths. Historical keyring keys can authenticate existing content but cannot serve as fresh destinations chosen by unsigned target rows. Existing authenticated envelopes for an inaccessible linked container remain reusable.
Document and blob content-key envelopes share one public format check: the
object-kind-specific AES-GCM suite, a canonical base64 12-byte IV, and a
canonical base64 48-byte wrapped 32-byte key plus authentication tag. The API
applies it to newly wrapped material, additionally requiring the wrapping
metadata to carry exactly suite and iv, and rejects a failing submission
before persistence. Clients apply the same check when reading a stored
envelope, minus that exact-key requirement, so an unrecognized metadata key can
never make a decryptable envelope unreadable. Every submitted envelope is held
to the strict submission shape, including retained wraps that link, unlink,
bind, rewrap, and sync writes resubmit verbatim; there is no retained-envelope
exemption. Stored projection reads remain available for diagnostics, but
malformed stored material cannot be resubmitted unchanged.
When a bundle submission is both malformed and stale, staleness usually wins:
the target hash and current-target checks run first, and only the target-set
comparison is ordered after the envelope shape check. A blob rewrap has no
preceding target-hash check — it recomputes the hash afterward — but its epoch
and coverage checks still run first.
A retired blob target that re-enters a bundle is the one case where what lands is not what was submitted: the server discards the resubmitted wrap and writes back the authentic stored one, because active key material cannot be replaced.
There are no legacy-suite or alternate-encoding paths. These checks validate structure; only a recipient with the KEK can authenticate the ciphertext and establish the recovered key. An authorized writer can still submit well-shaped, undecryptable material.
Clients and the API reject blob writes or attachment commits that omit targets for other active bindings.
Blob encrypted bytes and blob key packages are separate records. The encrypted
blob record carries the encrypted payload and its metadata: blob id, byte
length, content-key epoch, content record id, nonce-domain hash, metadata hash,
IV, suite, and ciphertext. The blob content-key bundle lives in
blob_content_key_epochs and
blob_content_key_targets, and attachment listing/bind responses return that
bundle alongside server-visible binding metadata.
Before promotion, the API hashes the staged object stream and validates it with the same binary-envelope parser SDK readers use: magic, bounded canonical JSON header, exact fields, suite/version, chunk layout, canonical 12-byte IV, and total encrypted length. The header buffer is bounded by 64 KiB plus the fixed prefix, and validation shares the existing hashing pass. The transaction also binds the parsed blob id, content-key epoch, record id, metadata hash, and nonce domain to the signed write header. Rejection rolls back promotion and binding. These checks reject malformed framing; an authorized writer can still submit well-shaped ciphertext that fails authentication or contains unwanted content.
That separation lets key packages change without restaging immutable blob bytes. If active bindings grow, the same blob content key can be wrapped to the expanded target set. If a container KEK rotates for an otherwise unchanged active binding set, the blob content key can be rewrapped to the new current container KEK targets. The latest stored bundle remains useful source material when a retained authorized client can still unwrap one of its targets and repair stale blob key packages.
Immutable blob bytes should keep one content-key epoch. A same-epoch rewrap is metadata maintenance; changing the blob content-key epoch requires replacing or re-encrypting the blob bytes so the encrypted record and write header remain bound to the key epoch that can decrypt them.
If access shrink means future use should not depend on the old blob key, the correct operation is blob replacement or content re-encryption, not silent content-key epoch rotation.
Local-First Create Intents
Local-first creation queues local intents, not fake access hashes. The SDK’s
durable outbox (local_intents) holds container creates, container moves, and
document placements, one intent per lane and subject, replayed in creation
order. An intent records its target, sync status, and, once a create is
accepted, the remote container id and metadata access-state hash.
Replay signs each access event against the latest verified remote manifests. A child under a local-only parent waits until the parent’s create is accepted:
- submit parent create against the latest verified remote parent manifest;
- receive authoritative parent manifest and KEK target state;
- submit child create signed against that authoritative parent result.
Documents created in a pending folder likewise wait for the folder’s create. The API accepts only server manifest hashes as access preconditions.
If the remote parent changed while offline, the client verifies the new parent manifest and re-signs the pending create against it.
Transparency And Checkpointing
Signed manifests prevent unauthorized mutation, but they do not alone prove latest-state freshness. The protocol uses local monotonic checkpoints first, with an optional transparency log for cross-client append-only proofs.
Minimum:
- clients durably pin each first-seen complete identity bundle;
- clients persist highest-seen principal policy version/hash per principal;
- clients persist highest-seen access manifest epoch/hash per object;
- clients refuse to move backwards without explicit recovery UX.
Production uses an exact durable identity pin (with provenance) plus two checkpoints:
PrincipalPolicyCheckpoint:principalType,principalId,version,stateHash.AccessManifestCheckpoint:objectKind,organizationId,objectId,epoch,manifestHash.
Verifier rules are fail-closed:
- lower version/epoch than the local checkpoint is rollback;
- same version/epoch with a different hash is equivocation;
- higher version/epoch is accepted only when the returned signed chain or predecessor proof extends the local checkpoint.
Stronger, as optional hardening:
- append identity states, principal policy heads, and object manifest heads to an append-only Merkle log;
- return inclusion and consistency proofs with API responses;
- let clients gossip or compare signed tree heads across devices.
The shared verifier’s transparency primitives use signed tree heads plus compact Merkle proofs. Identity state heads, principal policy heads, and access manifest heads are encoded as domain-separated transparency leaves. Inclusion proofs bind a leaf to a signed tree head. Consistency proofs bind a newer signed tree head to a previous pinned tree head. A first-contact client that has no pinned tree head, gossip peer, or witness can still be shown a self-consistent split view.
Public blockchain anchoring is optional. It can timestamp an aggregate log root and make server equivocation harder to hide, but it does not authorize object grants, solve first-contact identity trust, or prevent withholding. If used, anchor only aggregate tree roots, not object ids or metadata-bearing leaves.
Client Verification API
Verification lives in @tearleads/crypto, shared by the API and the client
SDK. Each entry point is async and returns
KeyingVerificationResult<T>: { ok: true, value: T } or
{ ok: false, error: KeyingVerificationError }.
verifyIdentityStateCheckpoint(input) -> VerifiedIdentityState;
verifyPrincipalPolicyBundle(input) -> VerifiedPrincipalPolicy;
verifyAccessManifest(input) -> VerifiedAccessManifest;
verifyContainerKekState(input) -> VerifiedContainerKekState;
deriveDocumentKekTargets(input) -> VerifiedDocumentKekTargets;
deriveBlobKekTargets(input) -> VerifiedBlobKekTargets;
verifyWriteHeader(input) -> VerifiedWriteHeader;
verifyTransparencyProof(input) -> VerifiedTransparencyProof;
verifyTransparencyInclusionProof(input) -> TransparencyTreeCheckpoint;
verifyTransparencyConsistencyProof(input) -> TransparencyTreeCheckpoint;
The two standalone transparency verifiers work on tree checkpoints (log id,
size, root) rather than signed heads, so a client can check that a served
head extends its persisted checkpoint without an unrelated leaf or inclusion
proof. verifyTransparencyProof composes them over signed heads.
These functions are deterministic and side-effect free. API routes use them for validation, but the security guarantee comes from clients running the same verification before encrypting or decrypting.
Programmable Proof Obligations
Keying treats verifier functions as the executable security specification. Each verifier returns a branded verified value, and later derivation functions only accept those verified values. This keeps unverified API JSON from flowing into encryption decisions by type accident.
Proof obligations:
VerifiedIdentityStateproves that an identity state head is not older than, and extends, the client’s local checkpoint. It does not bind that head to an identity trust root; first-contact trust remains a deployment boundary.VerifiedPrincipalPolicyproves the group/org policy chain is signed, contiguous, projection-bound, payload-bound, and key-epoch-correct for any shrink transition.VerifiedAccessManifestproves the object graph transition was signed by a user/device authorized under the previous verified manifest.VerifiedContainerKekStateproves each container KEK wrap is justified by the verified manifest, recipient key epoch, and parent KEK edge.VerifiedDocumentKekTargetsandVerifiedBlobKekTargetsprove the exact target set implied by the verified graph, not by an API-provided recipient list.VerifiedWriteHeaderproves encrypted content metadata commits to the object id, manifest hash, content key epoch, target hash, ciphertext hash, and writer identity; that the signer had write access under the committed manifest state; and that the ciphertext used a unique, domain-separated encryption record id, nonce, or subkey for the object/content-key epoch.VerifiedTransparencyProofproves the response leaf is included in the signed tree head and, when a previous tree head is supplied, that the newer tree head extends the pinned checkpoint.
This creates two classes of guarantees:
- Impossible without key compromise or broken first-contact trust: forging a grant, adding a recipient target, swapping a recipient key, or authorizing a write target that was not signed by an authorized principal.
- Detectable under checkpoint/transparency assumptions: replaying an old valid state or showing split views to different clients.
Withholding data, refusing proofs, delaying publication, and returning no state remain availability failures. Clients fail closed.
API Attack Outcomes
| API behavior | outcome |
|---|---|
| Adds unauthorized object grant | Missing authorized signed access event; client refuses to encrypt. |
| Changes group/org projection rows | Signed policy projection/hash mismatch; fail closed. |
| Reuses principal key after membership shrink | Policy transition rejected by verifier. |
| Swaps recipient public key | Identity/principal key fingerprint mismatch; fail closed unless first-contact trust was never established. |
| Omits a linked container target | Derived target hash differs; write/read verification fails. |
| Drops blob targets for another active binding | Blob target derivation from signed bindings catches mismatch. |
| Replays older manifest | Local highest-seen checkpoint or transparency consistency proof rejects when client has seen newer state. |
| Shows split views | Detectable through transparency consistency/gossip/witnessing; otherwise still an availability/equivocation gap. |
| Withholds state or proofs | Availability failure; clients fail closed. |
Storage State
Core server tables:
principal_statesprincipal_state_payloadsprincipal_membership_projectionprincipal_epoch_keysprincipal_member_envelopesaccess_eventsaccess_manifestsaccess_manifest_headscontainer_key_epochscontainer_key_wrapsdocument_content_key_epochsdocument_content_key_targetsdocument_content_write_headersblob_content_key_epochsblob_content_key_targetsblob_content_write_headers
Projection/cache tables include:
access_manifest_principal_head_projectionaccess_manifest_document_link_projectionattachment_bindings
Identity pins and monotonic checkpoints live in client-local storage; the server has no identity-state or transparency-log tables.
Projection tables are caches. The verifier must be able to reject a projection that does not match signed manifests.
Scaling Model
The design relies on not propagating access changes into every descendant document/blob row.
Expected costs:
| Operation | Expected write cost |
|---|---|
| Create container | O(1) plus parent manifest verification and one container KEK epoch. |
| Additive share on container | O(new recipient key wraps); no document/blob rewrite. |
| Revoke on container | O(remaining direct wrap targets for that container) plus one new container KEK epoch; descendants rekey lazily before future writes. |
| Move container | O(1) manifest/key update for moved container, unless policy chooses eager descendant rekey. |
| Link document to container | O(number of linked containers for that document), usually small. |
| Document write | O(number of linked containers for that document), not O(users). |
| Blob attach/write | O(number of active document/container target domains for that blob), not O(users). |
For tens of thousands of folders/documents and up to roughly 1000 users, this is viable if these constraints hold:
- Most documents are linked to a small number of containers.
- Most blobs have a small number of active bindings.
- Container KEKs inherit through parent KEK edges so ancestor additive shares do not require descendant key rewrites.
- Group grants target principal KEKs, not every user directly.
- The API maintains projection indexes for listing and search, but encryption decisions are made from verified manifests.
The scaling cliff to avoid is embedding expanded effective recipients or descendant document references inside container manifests. A container manifest should commit to direct grants and parent/key edges. It should not contain the full subtree or all linked documents.
With 1000 users, direct user grants on a container are still acceptable for
small shares, but organization-wide sharing should go through the reserved
Members group or another group principal key. Otherwise an additive grant can
still be O(1000) key wraps, which is manageable occasionally but should not
be the default hot path.
The main residual cost is lazy rekey after revocation. A subtractive change on a high-level container can force future writes in a large subtree to first materialize post-revocation descendant KEK epochs. The levels above a directly granted container are re-keyed with the rotation itself; everything else is first-writer-wins lazy materialization. The security rule is that future writes must not continue under a KEK chain reachable by the revoked principal.
Protocol Surface
The protocol covers:
- Canonical encodings and shared verifier APIs for Keying objects.
- Signed access event and manifest types for containers, document link sets, and attachment binding events.
- Principal-policy state storage, admin-signer checks, member envelopes, and local checkpoint verifier primitives.
- Container KEK epoch/wrap storage and target derivation.
- Document create/sync with document KEK targets and signed write headers.
- Signed document link/unlink manifests and multi-container target validation.
- Blob attachment binding/detach with blob KEK targets and staged blob write headers.
- Rotation-baseline markers and the attachment blob registry for signed write paths.
Deployment boundaries:
- Verifier outputs are required before encryption and decryption decisions; workflows that bypass them are outside the protocol guarantee.
- Offline structural creation depends on signed local intents until the server accepts authoritative manifests and KEK targets.
- Transparency log inclusion and consistency proofs are optional hardening around the core signed-state protocol.
- First-contact identity trust depends on identity links exchanged out of band, invitations, key transparency, or another external trust root.
Required Invariants
- An honest client can derive document/blob key targets without trusting API-authored recipient lists.
- A forged object grant cannot make an honest client encrypt to a new recipient without an authorized signature.
- Additive share on an ancestor does not rewrite descendant document/blob content-key bundles.
- Subtractive change prevents future writes under KEK epochs reachable by the removed subject.
- Multi-linked documents require all linked container KEK targets.
- Blob writes and attachment commits cannot drop targets for other active signed bindings.
- A read-only recipient cannot produce a content write that passes
verifyWriteHeader, even when they can decrypt the current content key. - Concurrent/offline writers cannot reuse an AEAD key/nonce pair for two different content records under the same content-key epoch.
- Local child creation under a local-only parent queues without inventing a server access hash.
- Clients enforce exact durable identity pins and monotonic checkpoints for principal policies and object manifests.
- Regression coverage includes malicious API fixtures for forged grants, swapped keys, omitted targets, stale manifests, split projection rows, and key-epoch reuse after membership shrink.