Transport and lakehouse projection

20. Iroh Transport

20.1. Role

Iroh is the preferred live network substrate.

It is not the authorization layer and not the persistence format.

20.2. Objects

Iroh-blobs SHOULD transfer exact encrypted PME object bytes.

SOP storage_id IS the iroh-blobs 32-byte BLAKE3 hash for those exact bytes; no translation or duplicate content-address namespace exists.

The object is encrypted before iroh receives it.

Verified streaming, download resumption, and verified range transfer are therefore inherited directly from iroh-blobs.

20.3. Control

MLS messages and small SSP/SOP metadata MAY travel over iroh application protocol streams.

20.4. Relay distinction

An iroh relay solves concurrent connectivity when direct paths fail.

The SSP mailbox solves asynchronous store-and-forward when peers are not online simultaneously.

Both roles remain.

20.5. Transport identity

Iroh transport identity SHOULD remain distinct from the stable SSP root identity.

SSP identity SHOULD authenticate the binding to authorized iroh endpoint identity.

This permits transport-key rotation without changing site_id.

21. Lakehouse Projection (Delta Lake)

21.1. Target decision

The interoperability target is Delta Lake, chosen for commercial reach: Delta is where enterprise analytical integration concentrates, and Delta UniForm keeps Iceberg readers reachable from a Delta base. DuckDB consumes the projection through its delta extension, so the local engine remains a first-class reader of the exported view.

Implementation is deferred to Phase 5. Two preparations are effective immediately because they cannot be retrofitted into immutable published objects: mandatory stable Parquet field IDs in SOP-PME-1 (Section 11.2) and Delta-convention partition-value encoding in object manifests (Section 12.2).

21.2. Projection only

The Delta representation is a derived interoperability surface.

It is not mesh consensus and not the authoritative SOP history.

21.3. Stable checkpoints

Projection occurs at explicit observed-state checkpoints rather than by mapping every newly discovered SOP object to a permanent global Delta version in HLC order.

Each checkpoint becomes one or a few Delta commits. A late-arriving mesh object lands in a later commit; previously published Delta versions are never rewritten. This matches Delta’s linear-history expectation exactly.

The export step SHOULD coalesce toward larger data files: SOP’s 16-64 MiB edge objects are an internal policy, while Delta consumers prefer larger files, and the checkpoint is the natural compaction point.

21.4. Transaction-log privacy

The Delta transaction log (_delta_log) is plaintext JSON on storage: schema, file paths, partition values, and per-file statistics are visible to whoever holds the target. The log MUST NOT be encrypted or restructured in a non-standard way; fighting the format forfeits the interoperability that motivates the projection.

Therefore, by default, a Delta projection targets trusted or user-controlled storage only. Per-file statistics MAY be written there, since they are valuable for consumer-side pruning. Projection to less-trusted storage is the explicit-release path of Section 21.6 and MAY strip per-file statistics (Delta tolerates stats-less add actions at the cost of consumer pruning).

21.5. Data-file re-encoding and delete materialization

Generic Delta readers expect standard Parquet and do not perform PME key resolution. A Delta projection therefore re-encodes data files rather than reusing PME object bytes: plaintext Parquet in the trusted target (protected by storage-level or client-side encryption of that target) or an explicit plaintext release. This is a per-checkpoint export cost, not a mesh cost.

Delta has no equality deletes, so SOP delete objects are materialized at checkpoint time by one of:

1. rewriting affected files without deleted rows (the initial implementation); or
2. emitting Delta deletion vectors (position-based), as an optimization.

Object immutability makes the equality-to-position resolution a one-time, cacheable computation per SOP object.

21.6. Explicit plaintext export

If the projection target cannot be treated as trusted user-controlled storage, producing the Delta view is an explicit export operation.

It MUST NOT happen as transparent background replication.

21.7. Target feature set

The projection pins a conservative Delta feature set so mainstream readers from recent years can consume the output:

• column mapping mode id (backed by SOP field IDs);
• deletion vectors optional, disabled until the optimization path ships;
• no writer features beyond those required by the above.

Current Delta specifics — deletion-vector encoding, column-mapping protocol versions, UniForm behavior — MUST be re-verified against the live specification at Phase 5 implementation time; this verification is a recorded open item.