GH-3710: opt-in in-memory idempotency guard for non-durable endpoints

[jeremydmiller]

Closes #3710

What

The durable inbox deduplicates incoming messages on the primary key of wolverine_incoming. Every non-durable mode had nothing at all — verified in the issue, and still true. #3708's EndpointMode.NativeAck makes that gap matter: it is at-least-once by design and produces a burst of duplicate deliveries — bounded by the broker's prefetch depth — on every rolling deploy, because whatever graceful drain cannot finish within the drain timeout is simply never settled.

This adds IIncomingIdempotencyGuard plus a generational default implementation, opted into per endpoint:

opts.ListenToRabbitQueue("webhooks")
    .ProcessInParallelWithNativeAcks()
    .PartitionProcessingByGroupId(PartitionSlots.Five)
    .WithInMemoryIdempotency(window: 5.Minutes(), maxTracked: 100_000); // both optional, these are the defaults

Default off. When it is off, the receivers pay one null check per delivery and nothing else.

Design notes

Where it sits. Same semantics as the durable path, minus the database: a duplicate is settled with the broker and dropped without reaching the handler, exactly like DurableReceiver.handleDuplicateIncomingEnvelope does when the inbox INSERT hits the primary key. Logged at Debug rather than Error — in a mode that never settles at receipt, redelivery is expected operational noise.

Bounding. An unbounded set of seen ids would be a leak on precisely the flood workload NativeAck exists for. Eviction is generational: two hash sets, both consulted on lookup, rotated on whichever comes first of window / 2 elapsing or maxTracked / 2 ids landing in the current generation. Hard ceiling of maxTracked, no per-entry timestamps, no LRU bookkeeping, O(1) everywhere. The price — stated in the docs — is that an id's lifetime is a range (at least window / 2, at most window, less under a flood) rather than a number.

In-flight vs processed. Tracked separately, so a duplicate arriving while the original is still executing is dropped rather than queued to run again. In-flight ids are generational too, purely as a leak stop.

Success vs failure. Only a delivery that reached a terminal the broker will not undo is remembered — Envelope.HasBeenAcked, which MessageContext.CompleteAsync sets on handler success and which a native dead-letter move also sets. Anything nacked, requeued, or deferred during a drain releases the id, because remembering it would suppress its own retry and turn a failure into a lost message.

Per-endpoint, and it survives receiver rebuilds. The guard is owned by the Endpoint, not the receiver: receivers are rebuilt on listener restart and back-pressure recovery, and those are exactly the moments that produce redeliveries. Global opt-in is the existing opts.Policies.AllListeners(x => x.WithInMemoryIdempotency()).

Honors DurabilitySettings.MessageIdentity — under IdAndDestination the key includes the destination, so the Modular Monolith shape (one process, same id at two listeners) still works.

Refused or inert where it is meaningless. Durable builds no guard and warns at startup (the inbox already does this, across restarts and across nodes). Local queues throw NotSupportedException where you call it, with a lazily-resolved fallback warning in ListenerConfigurationValidator.

JetStream and friends. Not implemented here, per the issue. The docs point at broker-side dedup where it exists — JetStream's Nats-Msg-Id duplicate window, SQS FIFO's MessageDeduplicationId, Pub/Sub's deduplication-id — as strictly better than an in-process guard, and note that this guard is chiefly for RabbitMQ classic/quorum queues, which have nothing.

The honest limit, and where it is documented

New "In-Memory Idempotency Guard" section in docs/guide/messaging/listeners.md, sitting between the NativeAck and Buffered sections so a user choosing a non-durable mode reads it in place, plus a row in the "which settings apply in which mode" matrix and pointers from the NativeAck shutdown bullet and the Buffered section.

It says out loud, in a warning block, that an in-memory guard does not survive a restart: the deploy that causes the redelivery burst also empties the guard on the node that starts up, a sibling node never knew, and a slot failover hands the queue to a node starting empty. The promise is at-least-once with best-effort dedup, not exactly-once; hard dedup means the durable inbox. It also notes that a duplicate slipping through is a cost/liveness issue rather than a correctness one for #3708's ordering guarantee — a duplicate still runs in its group's sequential lane.

Tests

• generational_idempotency_guard — rotation across both generations against an injected clock (survives one rotation, forgotten after two); bounded memory under 50,000 unique ids with a ceiling of 100; in-flight ids bounded even when nothing ever completes; in-flight duplicate rejected; released id re-runnable; both MessageIdentity modes; and a concurrency hammer asserting exactly one winner per id across 8 threads.
• in_memory_idempotency_guard_on_receivers — redelivery acked and never executed under NativeAck and under BufferedInMemory; both modes run the duplicate again with the guard off; concurrent duplicate dropped while the original is parked in the handler; and a deferred delivery not remembered, re-run through a rebuilt receiver on the same endpoint.
• listener_configuration_validation — opt-in only, tuning flows through to the guard, the defaults, the durable warning, and the local-queue refusal.

Broker-level redelivery integration testing on RabbitMQ (forcing a redelivery by closing the connection with unacked deliveries) is not in this PR.

Verification

• dotnet build wolverine.slnx -c Release -f net9.0 — clean, 0 warnings.
• Full CoreTests: 2566 total, 0 failed, 2 skipped (pre-existing).

🤖 Generated with Claude Code