Add EnableStrictStreamIdentityEnforcement flag for cross-partition stream id uniqueness#4293
Merged
jeremydmiller merged 1 commit intomasterfrom Apr 26, 2026
Merged
Conversation
…ream id uniqueness
Under UseArchivedStreamPartitioning the mt_streams primary key is forced to
include is_archived (PostgreSQL requires partition keys in unique constraints).
Archive physically moves a row from mt_streams_default to mt_streams_archived,
which leaves (id, FALSE) free for reuse and silently allows a fresh
StartStream to land on the identity of a previously archived stream — a
divergence from the non-partitioned mode, where flipping is_archived=TRUE on
the same row keeps the id occupied and the unique constraint still fires.
Add an opt-in flag, EnableStrictStreamIdentityEnforcement (default false),
that closes that gap. When enabled Marten creates a sibling, non-partitioned
mt_streams_identity table whose primary key is just the stream identity
(plus tenant_id under conjoined tenancy). The InsertStream codegen wraps the
mt_streams INSERT in a modifying CTE that pipes the identity columns into
mt_streams_identity in the same prepared statement, so a duplicate identity
raises a unique violation that InsertStreamBase.matches() recognizes and
TryTransform translates into ExistingStreamIdCollisionException — exactly
what the non-partitioned mode already does.
The flag is meant primarily for stores whose stream identity is generated
outside of Marten (most often user-supplied string keys); Marten-generated
Guids almost never need it, so leaving it off remains the right default.
Tests:
* strict_stream_identity_enforcement (8 facts):
- Guid + string \u00d7 partitioned + non-partitioned start/archive/start
- duplicate-without-archive still throws (sanity)
- append to existing stream still works (the sibling row is only
written on first append, not on every Append)
- flag-off + partitioned reuse still succeeds (documents the
pre-existing behavior the flag exists to fix)
* start_stream_with_id_of_previously_archived_stream (4 facts):
Standalone analysis baselines for both modes; useful regression
anchors and serve as user-facing documentation of the divergence.
Docs: new "Strict Stream Identity After Archive" section in
docs/events/archiving.md walks through the partitioned-PK mechanics, why
the flag exists, when it is and isn't worth turning on, and the trade-off
that mt_streams_identity grows monotonically (a separate "background
sweep / TTL" feature is left for a follow-up if there's actual demand).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Under
UseArchivedStreamPartitioningthemt_streamsPK must includeis_archived(PostgreSQL requires partition keys in unique constraints). Archive physically moves a row frommt_streams_defaulttomt_streams_archived, which leaves(id, FALSE)free for reuse — a freshStartStreamagainst a previously-archived id silently succeeds, diverging from the non-partitioned mode where the unique constraint still fires.This PR adds an opt-in
StoreOptions.Events.EnableStrictStreamIdentityEnforcementflag (defaultfalse) that closes that gap.When enabled:
mt_streams_identitytable is created whose PK is just the stream identity (plustenant_idunder conjoined tenancy).InsertStreamcodegen wraps themt_streamsINSERT in a modifying CTE that pipes the identity columns intomt_streams_identityin the same prepared statement, so a duplicate identity raises a unique violation thatInsertStreamBase.matches()recognizes andTryTransformtranslates into the sameExistingStreamIdCollisionExceptionyou'd get without partitioning.mt_streams_identity, so the protection spans the active / archived divide.The flag is primarily aimed at stores whose stream identity is generated outside of Marten (most often user-supplied string keys); Marten-generated Guids almost never need it, so leaving it off remains the right default.
Why a separate table (and not a unique index on
mt_streams.id)PostgreSQL forbids unique constraints on a partitioned table that don't include the partition key, so a literal
CREATE UNIQUE INDEX ON mt_streams (id)won't compile underUseArchivedStreamPartitioning. Adding the index per-partition doesn't help either, because archive removes the row from the active partition. A non-partitioned sibling table is the smallest mechanism that genuinely spans both partitions.We considered a "marker rows in
mt_streams_defaultwithkeep_until+ background sweep" alternative; that route is real but bigger (archive function rewrite, everyselect … from mt_streamsreader needs to filter markers out, plus a sweep entry point) and changes the semantics (allows id reuse after TTL). Left for a possible follow-up; the current flag's name reads as absolute and matches what the non-partitioned mode already guarantees.Test plan
strict_stream_identity_enforcement— 8 facts, all pass:start → archive → startthrowsAppendto an existing stream still worksstart_stream_with_id_of_previously_archived_stream— 4 facts, standalone analysis baselines for both partition modesDocumentation
docs/events/archiving.mdgets a new Strict Stream Identity After Archive section that walks through the partitioned-PK mechanics, when to use the flag (external string ids), and the explicit trade-off thatmt_streams_identitygrows monotonically (a "background sweep / TTL" feature is left as a possible follow-up).🤖 Generated with Claude Code