AI and training data

The specification makes no claims about AI, and sovm ships no model, no feature store, and no vector index. It is a data plane. But the properties it was designed for turn out to be close to what a serious AI data pipeline needs, and the reason is worth setting out precisely rather than asserting.

The bottleneck is governance, not capability

The easily-reachable training corpus — the public web — is largely spent. What remains genuinely valuable is private, continuously generated, and siloed: operational telemetry, sensor streams, device and process measurements, behavioural traces, access histories, and increasingly the decisions software agents take on someone’s behalf. It is the data that describes how systems, people and their agents actually behave over time, rather than what someone wrote down about it.

That data is also, almost by construction, the data nobody will hand over. An operator will not ship continuous plant telemetry to a third-party training service. A hospital will not export patient monitoring streams. An individual will not upload a decade of behavioural history. Not because the models would not benefit, but because the custody arrangement is unacceptable and often unlawful.

So the constraint on the most useful remaining data is not model capability, data volume, or compute. It is that using the data currently requires surrendering control of it. Any architecture that breaks that coupling widens what can be learned from.

Why time-series and telemetry specifically

Two reasons, and the second is newer than most data architecture assumes.

It is the substrate for most models that are not language models. Forecasting, anomaly detection, predictive maintenance, control, capacity planning, personalization, and causal inference all consume ordered observations over time. These are the workloads where a private dataset is not merely helpful but decisive, because the signal is specific to the system that produced it and cannot be substituted from a public corpus.

Time-series now has foundation models of its own. Pretrained, transferable time-series foundation models are an established category rather than a research curiosity. That changes the economics: an organization with a large private history of measurements can fine-tune rather than train from scratch, which makes that history considerably more valuable than it was when every model had to be built bespoke. It also raises the stakes on provenance, because a fine-tuned model inherits whatever was wrong with the data it saw.

Agents are now producing this shape of data too. A software agent acting on someone’s behalf emits exactly the same thing a sensor does — an ordered, immutable record of observations and decisions, continuously generated, and belonging to whoever it acted for. It is the newest instance of the pattern rather than a different problem, and it lands in the same place: valuable, private, and currently unusable without handing custody to whoever runs the model.

Telemetry is also, conveniently, the workload sovm handles best: high-volume, append-only, immutable once recorded. That is the case the design optimizes hardest and the one that needs no conflict resolution at all.

What sovm provides to an AI workload

Each of these is a property of the specification rather than a feature bolted on.

Compute goes to the data

The trust model does not require moving a dataset to whoever wants to train on it. An authorized node decrypts and computes locally; the machines storing and relaying the data never can. Storage, transport, and archival can therefore be delegated to infrastructure that is permanently outside the boundary, while training happens inside it.

This is the property that addresses the governance bottleneck directly, and it is why the edge posture matters here: the data does not have to leave the site, the device, or the jurisdiction that produced it in order to be learned from.

The objects are already the format ML tooling wants

SOP objects are encrypted Parquet. Once decrypted by an authorized node they are ordinary Parquet — consumable directly by Arrow, pandas, polars, DuckDB, and the standard framework data loaders, with column pruning and predicate pushdown intact.

There is no bespoke export step, no format conversion, and no second copy of the dataset maintained for the benefit of the training pipeline. That matters more than it sounds: the duplicate “ML copy” is usually where governance quietly breaks, since it is the copy nobody applied the policy to.

Datasets are reproducible by construction

A published object’s exact encrypted bytes must not change under the same identifier, and that identifier is the BLAKE3 hash of those bytes. So a training run can pin its input set by hash and a later run can prove it used the same data. Dataset versioning, normally a separate system layered on top with its own failure modes, is a consequence of how objects are addressed.

Deletion and compaction are represented as explicit supersession rather than mutation, so the state of a table at a point in time is reconstructible. That is what backtesting requires, and what avoids the class of evaluation error where a model is scored against data that leaked backwards into its training window.

Provenance is signed, not asserted

Every object carries an immutable producer signature over its manifest. The claim “this data came from that node” is verifiable rather than a database field somebody could have edited, and it survives an arbitrary number of untrusted hops between producer and consumer.

As training-data governance moves from good practice toward obligation — provenance records, dataset documentation, auditability of what a model was trained on – cryptographic attribution at the object level is a materially stronger position than a lineage table maintained by the pipeline that would also be the thing to misreport.

Training sets can be selected without decrypting everything

The encrypted catalog carries per-object pruning metadata: row counts, time ranges, partition values, min/max for configured columns. A training-set query can therefore select the objects it needs on time window, partition, or value range, and decrypt only those. Selectivity does not require bulk decryption.

Federated training is the natural consumer pattern

Because scope and placement are separate, and because the mesh already distributes key material only to authorized members, training across nodes without centralizing their data is the shape the architecture already has. Nodes hold their own partitions, an untrusted aggregator can carry traffic it cannot read, and no node needs to see another’s raw observations.

sovm does not implement this — it is a data plane, and federated learning is a layer above it. But it removes the part that usually blocks such a deployment, which is getting the data durably and verifiably into place without a trusted central store.

What sovm does not do for AI

This section matters as much as the one above, because the failure mode here is assuming an encrypted data plane confers privacy on the models trained from it. It does not.

• It does not make a model private. sovm provides no differential privacy, no secure aggregation, and no confidential computing. A model trained on sensitive data can memorize and leak it, and defending against that is a separate discipline operating at a different layer.
• A decrypting node sees plaintext. Training requires decryption, so a compromised training node is a full exposure of whatever it was authorized to read. Encryption at rest and in transit does not narrow that.
• It is not a feature store or a vector database. No embeddings, no online serving, no materialized feature views, no similarity index. Those are things you would build above it.
• It does not enable training across distrusting parties. A mesh spans one trust authority. Multi-party computation over data from mutually distrusting owners is a different problem and explicitly out of scope.
• Throughput on the training path is unmeasured. Encrypted Parquet carries non-trivial decryption overhead, and that cost lands on the hot path of a data loader. It is a gated property that has not been benchmarked, and it is the number most likely to determine whether a given training workload is practical.

The claim, stated plainly

sovm does not make your models better. It changes which data they are allowed to see, by removing the requirement that using data means surrendering custody of it – and it makes the resulting dataset reproducible and its provenance verifiable, which is increasingly the difference between a model you can deploy and one you cannot defend.

For high-volume telemetry and time-series in particular, that is the constraint that has been binding.