Object format (SOP-BLOB-1)

5. Requirements the format must satisfy

A media object differs from a Parquet object in one operational way: readers need byte ranges. A video player seeks; a viewer fetches the first bytes of an image progressively; a mobile device resumes a large download. SOP/1’s transfer layer already provides verified range transfer of ciphertext (section 11.7), so the encryption format must not squander it: it MUST be possible to decrypt an arbitrary chunk of the object without processing the chunks before it.

That requirement selects the construction. Chained streaming AEADs — where each chunk’s nonce or key depends on the previous chunk — provide fine sequential properties but forbid random access; a system that chooses one ends up rebuilding seek at another layer. SOP-BLOB-1 instead follows the STREAM shape used by the age file-encryption format: independent per-chunk AEAD with a nonce that is a pure function of the chunk index, plus an explicit final-chunk marker for truncation protection.

6. Construction

Profile: SOP-BLOB-1

AEAD:                 AES-256-GCM
key:                  DEK_object (fresh random 32 bytes, per SOP/1 section 7.4)
plaintext chunk:      1_048_560 bytes (1 MiB - 16)
ciphertext chunk:     1_048_576 bytes (exactly 1 MiB)
nonce (12 bytes):     big-endian 11-byte chunk counter || 1 flag byte
flag byte:            0x00 ordinary chunk, 0x01 final chunk
AAD:                  none
layout:               chunk_0 || chunk_1 || ... || chunk_final || key_ref (16 bytes)
key_ref:              random opaque 16 bytes, cleartext trailer

Rules:

1. Every chunk except the final one MUST contain exactly 1,048,560 plaintext bytes. The final chunk MUST contain at least 1 and at most 1,048,560 plaintext bytes and MUST be encrypted with flag byte 0x01.
2. The chunk counter starts at zero and increments by one per chunk. A decryptor MUST reject a stream whose final chunk does not carry the final flag (truncation), whose non-final chunk carries it (extension), or any chunk that fails authentication.
3. The DEK is used for no other purpose and for no other object (SOP/1 section 7.4). Nonce uniqueness is therefore structural: a fresh random key per object plus a counter nonce within it.
4. No additional authenticated data is used. Context binding does not rest on the AEAD: the object is content-addressed by its exact ciphertext, the producer manifest binds storage_id, and the AccessRecord binds the DEK to that manifest (acceptance). This mirrors the age design.
5. AES-256-GCM is chosen over a ChaCha20 construction to preserve the specification’s single-suite discipline: SOP/1’s COSE profile and PME profile already standardize on AES-GCM, and unused algorithm agility is treated as attack surface.

The 16-byte key_ref trailer serves the same purpose as the mandatory opaque key_metadata in SOP-PME-1: stray ciphertext found on a disk or a blind replica is self-indexing for forensics and recovery, at zero leakage cost because the value is random and never load-bearing. It is a trailer, not a header, for the alignment reason in the next section. Key resolution in the read path is always storage_id -> ObjectManifest + AccessRecord -> DEK; no reader may depend on the trailer.

7. The alignment rule

The transfer layer verifies ciphertext incrementally against a Merkle tree over fixed-size units and serves verified byte ranges at that unit’s granularity — 1 KiB in the reference transport. Its 16 KiB verification group governs how much of the hash tree is persisted, not how small a verified range may be. The AEAD layer decrypts in chunks. If the two grids do not align, every seek fetches and discards a partial unit on each edge.

SOP-BLOB-1 therefore fixes the ciphertext chunk, not the plaintext chunk: 1,048,560 plaintext bytes plus the 16-byte GCM tag give a ciphertext chunk of exactly 1 MiB — 64 verification groups, and 1,024 transfer units. Because chunks are concatenated from offset zero, every chunk begins on a boundary of both grids. The key_ref sits at the end of the object precisely so that it cannot shift this grid.

The reader MUST take the object’s length from the signed manifest’s plaintext_size and derive every chunk index and ciphertext span from the chunk grid; it MUST NOT take the object’s length from the transfer layer. A transport’s declared object length is not necessarily covered by the authentication of a served range — in the reference transport it is not — so a length obtained from a holder is unauthenticated input even when every byte that holder serves verifies. Range read, end to end:

1. The reader maps a plaintext range to chunk indices: first = floor(offset / 1_048_560), last = floor((offset + length - 1) / 1_048_560).
2. Chunk i’s ciphertext occupies object bytes [i * 1_048_576, (i + 1) * 1_048_576) — a whole number of transfer units under any group size that divides 1 MiB, requested as one verified range from any holder, including a blind replica.
3. Each received chunk is authenticated by the transfer layer against storage_id as it arrives, and then by GCM at the chunk boundary; the reader releases plaintext only after the chunk’s tag verifies.
4. Decrypted chunks are trimmed to the requested plaintext range.

A malicious or corrupt holder is detected after at most one transfer unit — 1 KiB in the reference transport; a forged chunk is detected at its tag. Seek granularity is 1 MiB of plaintext, which at typical video bitrates is a few seconds — acceptable for scrubbing without a second index. Applications that need finer-grained or format-aware streaming (adaptive bitrate, keyframe alignment) SHOULD produce a dedicated streaming derivative (derivatives) rather than shrinking the profile’s chunk size.

8. Size padding

Blob objects are exactly the case size padding exists for: a photo library’s object sizes are a fingerprint, and thumbnails — small, numerous, and countable — leak more than originals.

The writer MUST pad the plaintext with trailing zero bytes before encryption so that the final encrypted object length (chunks plus trailer) is the smallest achievable length greater than or equal to the Padmé target (SOP/1 section 11.8) computed over the unpadded encrypted length. Padding is therefore inside the AEAD: it cannot be stripped or extended without failing authentication, and untrusted storage observes only the padded size.

plaintext_size in the manifest (manifest profile) records the unpadded length; readers MUST truncate decrypted output to it.

An SOP-BLOB-1 writer MUST implement padding; there is no fail-open. Neither does SOP-PME-1, which requires the same of its writers. What differs is where the padding sits: PME has no natural slot for it, so it is carried as a reserved entry inside the encrypted footer and sized against a measured length model. A blob is a byte stream, so its padding is trailing plaintext and the target is reached by construction — no format fight, and no length model to be wrong about.

9. Storage identity, publication and immutability

Nothing changes from SOP/1:

• storage_id is the transfer-layer BLAKE3 hash of the exact encrypted object bytes, trailer and padding included (section 11.7). No second hash namespace exists. The unkeyed hash of the plaintext MUST NOT be exposed.
• Publication follows the section 13.2 pipeline with the Parquet encode and PME steps replaced by SOP-BLOB-1 encryption: durable local ciphertext first, then manifest, access record, and catalog entry; idempotently retryable; offline-created objects keep their DEKs under device-local protection and wrap at publication time (section 7.7).
• Published bytes never change under a storage_id; a corrected asset is a new object (section 12.5).