Implementation plan, gates and decisions

26. Recommended Implementation Plan

Phase 0 — conformance prototypes

Build isolated prototypes for:

1. OpenMLS group lifecycle;
2. SSP identity to MLS credential binding;
3. hardware-backed Signer integration;
4. encrypted OpenMLS storage provider;
5. random application domain KEK distribution over MLS;
6. SOP-PME-1 read/write with DuckDB;
7. external AccessRecord/wrapped-DEK catalog;
8. iroh-blobs transfer of exact encrypted Parquet bytes.

No production migration occurs in Phase 0.

Phase 1 — one APPEND domain

Implement a single non-critical APPEND dataset with:

• one control MLS group;
• one data-domain MLS group;
• one random DEK per object;
• SOP-PME-1;
• ciphertext BLAKE3 IDs;
• immutable producer manifest + encrypted access record;
• iroh transfer;
• local DuckDB querying.

Phase 2 — node membership lifecycle

Implement:

• MLS Add;
• Welcome delivery through asynchronous mailbox;
• MLS Remove;
• domain-group propagation;
• application KEK rotation on membership-security transitions;
• current-epoch publication checks;
• stale-node catch-up;
• rejection of removed-node publication;
• sequencing receipts, receipt gossip, and equivocation detection;
• hash-chained membership authorization records;
• domain genesis reset drill (Section 22.10), repeated as a recurring exercise thereafter.

Phase 3 — historical access

Implement:

• FORWARD_ONLY joins;
• FULL_HISTORY joins;
• COSE-signed historical KEK packages over authorized confidential channels;
• hardware-wrapped local historical key storage;
• catalog checkpoint objects and signed RecoveryRoot records (Section 12.7), gated by G10.

Phase 4 — APPEND_DELETE

Implement:

• encrypted equality-delete objects with permanent-tombstone semantics and durable compact tombstone-set representations (Sections 14.2, 14.4);
• optional data-object row-identity filters where enabled (Section 14.5);
• delete-aware DuckDB views;
• confidential compaction;
• supersession attestation, including prototyping and fixing the multiset logical digest (open item O1);
• deletion acknowledgement.

Phase 5 — interoperability

Implement the Delta Lake projection (Section 21) using explicit stable export checkpoints: trusted-storage default, checkpoint file coalescing, rewrite-based delete materialization first, and the conservative feature set of Section 21.7. Re-verify current Delta specifics at implementation time (open item O2).

27. Decision Gates

The following gates MUST pass before Revision 8 becomes a normative production protocol.

G1. OpenMLS mobile viability

Demonstrate correct operation on:

• iOS node hardware;
• Android node hardware;
• suspend/resume;
• process death;
• long offline periods;
• app upgrade;
• node restart.

Building for a target is insufficient; integration must be exercised.

G2. Secure hardware identity

Demonstrate the selected MLS credential/signature integration without silently exporting the SSP root identity private key.

G3. OpenMLS persistence security

Demonstrate:

• encrypted durable state;
• safe deletion behavior;
• no plaintext key backup;
• crash consistency;
• upgrade/migration behavior;
• iOS Keychain accessibility classes use an appropriate ThisNodeOnly policy for non-migratable key material;
• iOS key-store/support files are excluded from backup where appropriate and exclusion survives file replacement/rewrites;
• Android backup behavior is explicitly configured, including android:allowBackup, cloud-backup rules, node-transfer rules, and use of no-backup storage for sensitive files;
• restore/migration tests prove that copied application files alone do not recreate usable SOP/MLS key authority.

G4. Membership/fork behavior

Exercise:

• simultaneous proposals;
• concurrent commits;
• stale commits;
• delayed Welcome;
• offline member catch-up;
• removed offline member;
• blind relay CAS sequencing;
• simultaneous Commit race with exactly one accepted winner;
• second-Commit rejection for the same MLS epoch;
• relay outage with queued membership changes;
• semantically invalid accepted Commit followed by client rejection and recovery;
• fork detection/recovery for states created outside the normal sequenced path;
• sequencer bootstrap for a newly created group;
• rejection of concurrent dual-sequencer operation and correct epoch transfer on sequencer migration;
• receipt-based sequencer state-loss recovery: recovery from concordant receipt-chain evidence, fail-closed on divergent valid chains, and genesis-reset fallback when no usable receipt evidence survives;
• committer death: MLS epoch advanced but the KEK-distribution message never arrives, followed by recovery through a fresh security rotation and voiding of the undistributed generation;
• sequencer equivocation detection from two valid receipts for the same (group_id, epoch) with different commit hashes, followed by sequencer migration and state audit;
• receipt-chain continuity verification across sequencer migration.

G5. PME interoperability

Demonstrate that SOP-PME-1 files can be:

• written;
• transferred unchanged;
• read by target DuckDB versions;
• selectively scanned;
• compacted;
• verified after corruption tests;
• written with a random non-semantic AAD prefix;
• Padmé-padded while remaining byte-for-byte valid PME/Parquet;
• written with stable Parquet field IDs on every column;
• written with an always-present random 16-byte key_ref;
• imported into iroh-blobs with storage_id exactly equal to the returned iroh-blobs hash.

G6. PME performance

Benchmark representative datasets on:

• desktop x86-64;
• Apple Silicon;
• Android ARM;
• iPhone/iPad ARM.

Measure:

• ingestion throughput;
• encrypted write overhead;
• selective scan overhead using manifest/catalog-first pruning before PME open;
• false-positive/open rate from manifest/catalog pruning;
• APPEND_DELETE delete-filter pruning effectiveness, including optional data-object row-identity filters, their false-positive rates, and manifest/companion-object size impact;
• full scan overhead;
• compaction;
• memory use;
• battery impact;
• effect of 16, 32, and 64 MiB object targets;
• effect of representative Parquet row-group sizes;
• CPU/platform AES-GCM acceleration availability and impact, including ARMv8 AES/PMULL or equivalent acceleration on minimum supported Android hardware.

G7. Revocation semantics

Verify through adversarial tests that a removed node cannot:

• receive current post-removal domain KEK generations;
• receive unauthorized historical key packages;
• publish new accepted manifests/access records;
• decrypt future objects.

Additionally verify through adversarial tests that a FORWARD_ONLY member cannot unwrap or decrypt any object predating its admission boundary, including after arbitrary rewrap and KEK-generation-retirement activity in the domain (Section 7.6).

Also verify documentation clearly states that previously authorized historical data cannot be forcibly forgotten.

G8. Metadata leakage review

Confirm that untrusted infrastructure does not receive semantic:

• table names;
• schema names;
• domain names;
• plaintext statistics;
• row keys;
• key names.

G9. COSE interoperability and misuse resistance

Demonstrate:

• AccessRecord/wrapped-DEK COSE_Encrypt0 conformance;
• producer-manifest COSE_Sign1 conformance;
• access-record issuer-signature and outer COSE_Encrypt0 conformance;
• historical key-history COSE_Sign1 conformance;
• deterministic CBOR test vectors;
• protected-header and algorithm validation;
• external-AAD context binding;
• nonce uniqueness under A256GCM;
• rejection of unknown critical headers;
• rejection of algorithm downgrade or cross-context replay.

G10. Disaster recovery

Demonstrate, on representative hardware:

• periodic catalog checkpoint objects and encrypted+signed RecoveryRoot records are produced and placed on blind replicas under ordinary placement policy;
• the protected node recovery/key store retains the latest known RecoveryRoot or its content hash without depending on an iroh-local tag;
• full recovery of a domain dataset from a blind archive plus a bare node recovery/key store only: RecoveryRoot resolved, checkpoint fetched by checkpoint_storage_id, checkpoint access record opened with retained KEK history, catalog restored, objects resolved and queried;
• recovery from a stale but valid RecoveryRoot produces a consistent older catalog and validates any later root chain supplied by the archive adapter;
• domain genesis reset end-to-end, including receipt-chain restart and old group retirement;
• sequencer state-loss recovery succeeds from concordant signed receipt evidence and fails closed on divergent valid receipt chains; and
• recovery fails closed, with a clear user-facing state, when any of the three required components (ciphertext, manifest/access metadata, KEK history) is absent.

28. Decisions and Remaining Open Items

Revisions 4 through 8 resolved the previously open architectural questions. The decisions are recorded here with pointers to their normative sections; the section text is authoritative.

D1. Membership authorization (was Q1)

Membership Commits are authorized by portable COSE-signed SSP authorization artifacts with monotonic generations; local SSP-policy freshness is not an acceptance dependency. Section 5.7.

D2. Credential binding transcript (was Q2)

Fixed deterministic-CBOR COSE_Sign1 transcript under the SSP root identity key, with monotonic credential_generation instead of wall-clock validity and ciphersuite binding. Section 5.3.

D3. Credential strategy (was Q3)

One SSP-bound credential per node, reused across all SOVM groups; pseudonymous per-group credentials are deferred to a multi-user future. Section 5.3.

D4. KEK history retirement (was Q4)

Lazy retirement: no proactive rewrap campaigns; compaction migrates data forward; a generation is retired when unreferenced and the Section 7.6 entitlement condition holds; tombstones are kept; active retirement is incident response only. Section 7.6.

D5. Supersession attestation and dispute (was Q5)

COSE attestation and dispute record shapes are fixed, built on an order-independent keyed multiset digest of logical content. Revision 6 introduced the split into verified GC (adversarial guarantee via a second decrypting verifier) and single-verifier GC (operational guarantee only, explicitly trusting the sole full replica), with blind replicas retaining superseded ciphertext until attestation by default. Section 18.5. The digest construction itself remains open (O1).

D6. PME key metadata (was Q6)

A random opaque 16-byte key_ref is always written and never load-bearing; resolution is always storage_id -> ObjectManifest + current AccessRecord -> DEK. Sections 11.2, 11.5, 12.2-12.3.

D7. Offline pre-removal data (was Q7)

Fail-closed rejection stands; mitigation is a pre-removal synchronization check in the removal ceremony plus explicit re-admission. Section 9.5.

D8. Domain cardinality (was Q8)

Initial taxonomy is CONTROL plus GENERAL; new domains are created on real scope asymmetry. Table-to-domain migration leaves PME bytes and the immutable producer ObjectManifest unchanged and issues new target-domain AccessRecords; it cannot revoke previously learned source-domain data. Section 6.4.

D9. Delete format (was Q9)

Exactly one encoding: encrypted equality-delete Parquet objects. Revision 6 replaced Iceberg-style sequence-number semantics with permanent identity tombstones; Revision 7 makes the logical tombstone set non-expiring while allowing delete-object bytes to compact into durable tombstone-set representations. Position deletes and deletion vectors remain a gated Phase 4+ optimization. Sections 14.2-14.5.

D10. Lakehouse target (was Q10)

Delta Lake, chosen for commercial reach, deferred to Phase 5. Immediate preparations: mandatory stable Parquet field IDs (Section 11.2) and Delta-convention partition values in object manifests (Section 12.2). Trusted- storage-only default for the plaintext transaction log; file re-encoding and delete materialization per Sections 21.4-21.5.

D11. Group recovery (was Q11)

The domain genesis reset, backed by restated invariant 22 (loss of all MLS protocol state is never data loss while one recoverable copy exists of the ciphertext, its manifest/access metadata, and applicable KEK history), catalog checkpoints plus signed RecoveryRoot records (Section 12.7), receipt-based sequencer recovery (Section 22.6), a recurring Phase 2 drill, and disaster-recovery Gate G10. Section 22.10.

D12. Manifest/access split

Object provenance and current decryption authority are separate records. ObjectManifest is immutable and producer-signed; AccessRecord is independently issuer-signed, domain-encrypted, and replaceable/augmentable. Rewraps and domain migration never require the original producer to re-sign. Sections 7.5-7.7 and 12.2-12.3.

D13. Permanent tombstone knowledge

APPEND_DELETE tombstones do not expire based on expected offline duration. Delete-object bytes may compact, but the logical deleted-identity set remains represented until a future explicit identity-namespace closure makes older objects inadmissible. Section 14.4.

D14. Receipt-based sequencer recovery

Sequencer state loss is recovered from concordant signed receipt-chain evidence plus current MLS-state attestation. Divergent valid receipt histories fail closed as proof of equivocation; absence of usable receipts falls back to domain genesis reset. Section 22.6.

D15. SOVM RecoveryRoot

Catalog-checkpoint bootstrap is named by an encrypted+signed, content-addressed SOVM RecoveryRoot, not by an iroh-blobs tag. Storage adapters may use local tags or object-store keys to pin roots, but those locators are not protocol state. Section 12.7.

D16. Baseline freeze (Revision 8)

G4 tests receipt-based sequencer recovery; entitlement_class is defined as the recorded Section 8 entitlement mode of a grant; AccessRecord carries issued_hlc for audit ordering with issuing authority anchored structurally in KEK-generation possession plus the Section 5.7 authorization chain. Revision 8 is the frozen implementation baseline; further design change requires a successor RFC.

Remaining open items

O1. Multiset logical digest construction. The exact keyed order-independent digest used by supersession verification (Section 18.5) requires prototyping and a fixed specification before Phase 4 APPEND_DELETE compaction ships.

O2. Delta specification verification. Deletion-vector encoding, column-mapping protocol versions, and UniForm behavior evolve; they MUST be re-verified against the live Delta specification at Phase 5 implementation time (Section 21.7).