Round-trip Envelope.DeduplicationId, and let a ping reach a FIFO destination (GH-3793)

[jeremydmiller]

Fixes #3793.

The bug

EnvelopeSerializer wrote GroupId and PartitionKey but never DeduplicationId. Any envelope that went through durable storage — node recovery after a restart, or reassignment while a sending agent is latched — came back with DeduplicationId == null.

Publishing that recovered envelope to an SNS/SQS FIFO destination that doesn't have ContentBasedDeduplication turned on is then rejected deterministically:

Invalid parameter: The topic should either have ContentBasedDeduplication enabled or MessageDeduplicationId provided explicitly

So it retries forever, or dead-letters if a SendingFailure policy is configured. The exact outage the durable outbox exists to survive was the one case that failed — and because messages on the in-memory fast path publish fine, it looks like a consequence of the outage itself.

The reporter's diagnosis and proposed fix were exactly right. The set-side was covered by DeliveryOptionsTests; the storage round-trip never was.

The fix

deduplication-id is now written and read symmetrically with group-id. It also joins ReservedHeaderKeys — reserved_header_key_filtering.every_key_promoted_by_the_reader_is_in_the_reserved_set already guards that invariant and would have failed otherwise (GH-3408).

Backward compatible both ways: previously-stored envelopes simply lack the key, and the key-value format lets older readers skip an unknown one.

Two related AWS defects

Both reproduced against LocalStack, which returns the same validation errors as real AWS here:

publish	before
standard topic + MessageGroupId	OK
standard topic + MessageDeduplicationId	rejected — "not valid for this topic type"
FIFO topic, group but no dedup id	rejected — the error above
FIFO topic, group + dedup id	OK

1. The circuit-resume ping could never reach a FIFO destination. Envelope.ForPing carries a GroupId but no DeduplicationId, so the probe hit the same validation — a latched sender could never unlatch on a FIFO queue/topic no matter how healthy the broker was. Both AWS transports now fall back to the envelope id, for pings only. That's also the semantic we want: every ping body is the same four bytes, so content-based deduplication would happily collapse consecutive probes into one.

2. SNS mapped MessageDeduplicationId unconditionally, and a standard topic rejects that parameter outright. It's now gated on the topic type the way AmazonSqsQueue already gated it.

Ordinary envelopes with no DeduplicationId are untouched, so FIFO destinations relying on ContentBasedDeduplication keep behaving exactly as they do today.

Verification

Both new integration tests were confirmed red before the fix — the ping one with the issue's verbatim error message.

• CoreTests — 2231 passed
• Wolverine.AmazonSns.Tests — 126 passed
• Wolverine.AmazonSqs.Tests — 264 passed
• dotnet build wolverine.slnx -c Release — clean

New coverage: serialization_and_deserialization.deduplication_id, sqs_fifo_deduplication_id_3793, sns_fifo_deduplication_id_3793, and fifo_topic_publishing_3793 (end-to-end against a real FIFO topic with ContentBasedDeduplication=false).

A 5.0 backport is coming in a separate PR, per the reporter's request.

🤖 Generated with Claude Code

https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA