Skip to content

[codex] Scope JetStream dedupe by subject - #22

Draft
SadPossum wants to merge 1 commit into
devfrom
codex/jetstream-subject-dedupe
Draft

[codex] Scope JetStream dedupe by subject#22
SadPossum wants to merge 1 commit into
devfrom
codex/jetstream-subject-dedupe

Conversation

@SadPossum

Copy link
Copy Markdown
Owner

What changed

  • derive JetStream Nats-Msg-Id as a fixed 64-character SHA-256 digest of the normalized subject plus canonical outbox GUID
  • preserve same-subject/same-ID broker de-duplication while allowing different subjects and event types with the same GUID to coexist in one stream
  • add deterministic unit vectors and a real NATS provider scenario covering same-subject replay, same-GUID/different-subject publication, and duplicate-window expiry
  • document the rollout overlap and the remaining producer/outbox identity boundary

Root cause and impact

The adapter previously used the GUID alone as Nats-Msg-Id. JetStream de-duplication is stream-wide, so two different subjects sharing an EventId collided even when they represented different public events. This change is transport-internal: it adds no public API or persistence schema.

Module outbox tables are unchanged and remain keyed by GUID alone. Producers must still generate globally collision-resistant integration-event IDs because same-ID rows cannot coexist inside one module outbox even when their subjects differ.

During rollout, an outbox row first published with the old GUID-only key and then retried with the new subject-scoped key can be stored once under each key. Drain the publishing backlog before upgrade when that overlap is unacceptable; consumer inbox idempotency remains required.

Validation

  • dotnet restore Gma.Framework.slnx
  • dotnet build Gma.Framework.slnx --no-restore -m:1 -nr:false: 0 warnings, 0 errors
  • ./eng/test-fast.ps1 -NoBuild: 1,125 passed
  • ./eng/test-docker.ps1 -NoBuild: 3 passed, including the real NATS duplicate-window proof
  • changed-file dotnet format style and dotnet format analyzers verification
  • solution synchronization, repository security, repository release, and git diff --check
  • transitive NuGet vulnerability audit: no vulnerable packages

Formatting baseline note

A solution-wide whitespace dotnet format --verify-no-changes is not a valid green signal in this Linux checkout: the repository .gitattributes normalizes tracked C# files to LF while .editorconfig requires CRLF, producing ENDOFLINE diagnostics across untouched baseline files. Changed-file style and analyzer verification passed, and the zero-warning build enforces the configured code-style analyzers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant