Pulsar: producer deduplication (#3185)#3212
Merged
Merged
Conversation
Opt-in producer→broker deduplication on a sending endpoint:
opts.PublishMessage<OrderPlaced>().ToPulsarTopic(topic).EnableDeduplication();
The producer is created with a stable producer name and stamps a monotonic
per-message sequence id (Pulsar dedups on producer-name + sequence-id). The
sequence id is cached by Envelope.Id, so a repeat send of the same envelope
(e.g. an outbox resend after a transient failure) reuses its id and the broker
discards the duplicate, while a brand-new envelope always gets the next id and
is delivered. The counter is seeded from the clock so a fresh producer session
never reuses ids from a prior one (DotPulsar 5.1.2 exposes no broker
LastSequenceId to resume from).
This is producer->broker dedup only, not end-to-end exactly-once, and requires
broker deduplication to be enabled on the namespace/topic. Pulsar transactions
are NOT exposed by DotPulsar 5.1.2 (only exception types) and are deferred /
out of scope, matching the epic's declared non-goal.
Tests: resending the same envelope is deduplicated by the broker (a distinct
envelope is delivered) verified against a real broker; config unit test. Docs:
Producer Deduplication section noting the producer->broker-only boundary.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 23, 2026
This was referenced Jun 23, 2026
Closed
This was referenced Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3185. Final issue of the Re-Evaluate Pulsar Integration epic — also closes #3176.
(Rebased replacement for #3211, which became conflicting after the JSON-schema PR #3210 merged; identical content, conflicts with #3210 resolved so schema + dedup coexist.)
Opt-in producer→broker deduplication on a sending endpoint:
How it works
producer-name + sequence-id).Envelope.Id, so a repeat send of the same envelope (e.g. an outbox resend after a transient failure) reuses its id and the broker discards the duplicate, while a brand-new envelope always gets the next id and is delivered.LastSequenceIdto resume from).Scope
Acceptance criteria
resending_the_same_envelope_is_deduplicated_by_the_broker: resent envelope dropped, distinct one delivered → 2 messages from 3 sends) against a real broker.Tests / build
pulsar_producer_dedup: integration + config unit test — green against a Testcontainers broker.wolverine.slnxbuilds clean in Release (with Pulsar: JSON schema support with broker-side registration (#3183) #3210's schema work merged in).10th and final child of epic #3176. With this merged the epic is complete: #3186, #3178, #3181, #3177, #3179, #3180, #3184, #3182, #3183, #3185.
🤖 Generated with Claude Code