Skip to content

✨ feat(replication): version the operation envelope#611

Merged
gaborbernat merged 1 commit into
tox-dev:mainfrom
gaborbernat:feat/versioned-replication-op-envelope-503
Jul 18, 2026
Merged

✨ feat(replication): version the operation envelope#611
gaborbernat merged 1 commit into
tox-dev:mainfrom
gaborbernat:feat/versioned-replication-op-envelope-503

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

The availability contracts express staleness and recovery as a serial frontier over the peryx-replication change serial, yet the change stream carries no per-operation provenance a follower can reason about before it applies an operation. A Change holds a serial and an opaque payload. It does not say which schema version encoded it, which primary authored it, which authority generation was in force, or what kind of mutation it is, and nothing bounds an untrusted peer's bytes before they parse. Storage durability and a later transport both need to depend on the stream, so replication first needs a stable, versioned envelope around each operation.

OperationEnvelope wraps the existing version-1 Change instead of opening a second replication stream. It adds a schema_version, the source identity, an AuthorityEpoch fencing token that a failover advances, an OperationKind naming the mutation, and an optional W3C TraceContext. An operation's identity is the (source, epoch, serial) triple: stable across replay, idempotent to apply, and safe to log. encode and decode round-trip through JSON. decode treats its input as untrusted: it rejects bytes past a size or nesting-depth limit before parsing, requires a schema version within SUPPORTED_SCHEMA_VERSIONS, and validates the traceparent when a trace context is present. SchemaVersion::negotiate is the version-negotiation surface, returning the newest version two ends share.

Two compatibility rules keep a rolling upgrade between adjacent schema versions safe. Decoding ignores fields it does not recognize, so a newer producer that adds a field stays readable by an older consumer inside the supported window, and it refuses any version outside that window instead of guessing at a schema it cannot model. Debug and Display render the operation kind, version, and identity, and they omit the payload bytes and metadata values, so a log line carries what an operator needs to trace an operation without its contents. A local benchmark reports the encode and decode cost and allocation count for a representative wheel upload and registry manifest push, holding metadata replication clear of becoming a blob transport.

Persistence, transport, and applying operations stay downstream and out of scope; this change defines the wire and persistence type, its bounded encode and decode, and its version-negotiation surface.

Closes #503

@gaborbernat gaborbernat added the enhancement New feature or request label Jul 18, 2026
@read-the-docs-community

read-the-docs-community Bot commented Jul 18, 2026

Copy link
Copy Markdown

@codspeed-hq

codspeed-hq Bot commented Jul 18, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 32 untouched benchmarks
⏩ 94 skipped benchmarks1


Comparing gaborbernat:feat/versioned-replication-op-envelope-503 (705408f) with main (9adc775)

Open in CodSpeed

Footnotes

  1. 94 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@gaborbernat
gaborbernat enabled auto-merge (squash) July 18, 2026 04:19
Wrap the version-1 Change in an OperationEnvelope that carries the
schema version, source, authority epoch, operation kind, and an
optional W3C trace context, keeping the identity as the
(source, epoch, serial) triple over the existing change serial.

Bounded encode/decode rejects oversized or over-nested peer bytes,
enforces the supported schema-version window, tolerates unknown
fields for a rolling upgrade, and redacts payload from Debug/Display.
@gaborbernat
gaborbernat force-pushed the feat/versioned-replication-op-envelope-503 branch from f03056b to 705408f Compare July 18, 2026 04:21
@gaborbernat
gaborbernat merged commit a59b93d into tox-dev:main Jul 18, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Define a versioned replication operation envelope

1 participant