Skip to content
sovm.io

Object format (SOP-BLOB-1)

1. Requirements the format must satisfy

Section titled “1. Requirements the format must satisfy”

A media object differs from a Parquet object in one operational way: readers need byte ranges. A video player seeks; a viewer fetches the first bytes of an image progressively; a mobile device resumes a large download. SOP/1’s transfer layer already provides verified range transfer of ciphertext (section 11.7), so the encryption format must not squander it: it MUST be possible to decrypt an arbitrary chunk of the object without processing the chunks before it.

That requirement selects the construction. Chained streaming AEADs — where each chunk’s nonce or key depends on the previous chunk — provide fine sequential properties but forbid random access; a system that chooses one ends up rebuilding seek at another layer. SOP-BLOB-1 instead follows the STREAM shape used by the age file-encryption format: independent per-chunk AEAD with a nonce that is a pure function of the chunk index, plus an explicit final-chunk marker for truncation protection.

2. Construction

Section titled “2. Construction”
Profile: SOP-BLOB-1


AEAD:                 AES-256-GCM
key:                  DEK_object (fresh random 32 bytes, per SOP/1 section 7.4)
plaintext chunk:      1_048_560 bytes (1 MiB - 16)
ciphertext chunk:     1_048_576 bytes (exactly 1 MiB)
nonce (12 bytes):     big-endian 11-byte chunk counter || 1 flag byte
flag byte:            0x00 ordinary chunk, 0x01 final chunk
AAD:                  none
layout:               chunk_0 || chunk_1 || ... || chunk_final || key_ref (16 bytes)
key_ref:              random opaque 16 bytes, cleartext trailer

Rules:

  1. Every chunk except the final one MUST contain exactly 1,048,560 plaintext bytes. The final chunk MUST contain at least 1 and at most 1,048,560 plaintext bytes and MUST be encrypted with flag byte 0x01.
  2. The chunk counter starts at zero and increments by one per chunk. A decryptor MUST reject a stream whose final chunk does not carry the final flag (truncation), whose non-final chunk carries it (extension), or any chunk that fails authentication.
  3. The DEK is used for no other purpose and for no other object (SOP/1 section 7.4). Nonce uniqueness is therefore structural: a fresh random key per object plus a counter nonce within it.
  4. No additional authenticated data is used. Context binding does not rest on the AEAD: the object is content-addressed by its exact ciphertext, the producer manifest binds storage_id, and the AccessRecord binds the DEK to that manifest (acceptance). This mirrors the age design.
  5. AES-256-GCM is chosen over a ChaCha20 construction to preserve the family’s single-suite discipline: SOP/1’s COSE profile and PME profile already standardize on AES-GCM, and unused algorithm agility is treated as attack surface.

The 16-byte key_ref trailer serves the same purpose as the mandatory opaque key_metadata in SOP-PME-1: stray ciphertext found on a disk or a blind replica is self-indexing for forensics and recovery, at zero leakage cost because the value is random and never load-bearing. It is a trailer, not a header, for the alignment reason in the next section. Key resolution in the read path is always storage_id -> ObjectManifest + AccessRecord -> DEK; no reader may depend on the trailer.

3. The alignment rule

Section titled “3. The alignment rule”

The transfer layer verifies ciphertext incrementally in fixed groups — 16 KiB in the reference transport — and serves verified byte ranges at that granularity. The AEAD layer decrypts in chunks. If the two grids do not align, every seek fetches and discards partial groups on both edges.

SOP-BLOB-1 therefore fixes the ciphertext chunk, not the plaintext chunk: 1,048,560 plaintext bytes plus the 16-byte GCM tag give a ciphertext chunk of exactly 1 MiB — 64 transfer groups. Because chunks are concatenated from offset zero, every chunk begins on a group boundary. The key_ref sits at the end of the object precisely so that it cannot shift this grid.

Range read, end to end:

  1. The reader maps a plaintext range to chunk indices: first = floor(offset / 1_048_560), last = floor((offset + length - 1) / 1_048_560).
  2. Chunk i’s ciphertext occupies object bytes [i * 1_048_576, (i + 1) * 1_048_576) — whole transfer groups, requested as one verified range from any holder, including a blind replica.
  3. Each received chunk is authenticated by the transfer layer against storage_id in 16 KiB steps and then by GCM at the chunk boundary; the reader releases plaintext only after the chunk’s tag verifies.
  4. Decrypted chunks are trimmed to the requested plaintext range.

A malicious or corrupt holder is detected after at most one transfer group; a forged chunk is detected at its tag. Seek granularity is 1 MiB of plaintext, which at typical video bitrates is a few seconds — acceptable for scrubbing without a second index. Applications that need finer-grained or format-aware streaming (adaptive bitrate, keyframe alignment) SHOULD produce a dedicated streaming derivative (derivatives) rather than shrinking the profile’s chunk size.

4. Size padding

Section titled “4. Size padding”

Blob objects are exactly the case size padding exists for: a photo library’s object sizes are a fingerprint, and thumbnails — small, numerous, and countable — leak more than originals.

The writer MUST pad the plaintext with trailing zero bytes before encryption so that the final encrypted object length (chunks plus trailer) is the smallest achievable length greater than or equal to the Padmé target (SOP/1 section 11.8) computed over the unpadded encrypted length. Padding is therefore inside the AEAD: it cannot be stripped or extended without failing authentication, and untrusted storage observes only the padded size.

plaintext_size in the manifest (manifest profile) records the unpadded length; readers MUST truncate decrypted output to it.

Unlike SOP-PME-1 — where padding must be smuggled through a container format that offers no natural slot, and the capability is allowed to fail — there is no format fight here. An SOP-BLOB-1 writer MUST implement padding; there is no fail-open.

5. Storage identity, publication and immutability

Section titled “5. Storage identity, publication and immutability”

Nothing changes from SOP/1:

  • storage_id is the transfer-layer BLAKE3 hash of the exact encrypted object bytes, trailer and padding included (section 11.7). No second hash namespace exists. The unkeyed hash of the plaintext MUST NOT be exposed.
  • Publication follows the section 13.2 pipeline with the Parquet encode and PME steps replaced by SOP-BLOB-1 encryption: durable local ciphertext first, then manifest, access record, and catalog entry; idempotently retryable; offline-created objects keep their DEKs under device-local protection and wrap at publication time (section 7.7).
  • Published bytes never change under a storage_id; a corrected asset is a new object (section 12.5).