GH-3709: ProcessInParallelWithNativeAcks() on the global partitioned topology

[jeremydmiller]

Toward #3709 — the topology half. This is a partial implementation and #3709 must stay open after it merges; see Scope below.

Deliberately no linking keyword anywhere in this description. An earlier draft phrased the scope note as "does not ​close" immediately followed by the issue reference; GitHub's parser matched that as a closing keyword — negation and all — and linked this PR to auto-close the issue on merge. Please do not add one.

Partitioned clustering with no database: each slot listener settles its own broker deliveries and shards into sequential lanes by group id in memory, instead of bridging into a companion durable local queue and paying an inbox insert plus a mark-as-handled per message.

opts.MessagePartitioning.GlobalPartitioned(topology =>
{
    topology.ByMessage<WebhookEvent>(x => x.EntityId);
    topology.UseShardedRabbitQueues("webhooks", 5);
    topology.ProcessInParallelWithNativeAcks(); // NEW
});

Why this needed a new method rather than Mode(EndpointMode.NativeAck)

The default topology hands each external listener to GlobalPartitionedReceiverBridge, which forwards into a companion local queue where the partitioned execution actually happens. That is exactly why #4032 made Mode() reject NativeAck: a local queue has no broker delivery to settle, so the ack would have nothing to ack against.

Opting in removes the companion topology and the bridge, so the premise no longer holds. That guard is replaced, not deleted — Mode(NativeAck) still throws, because it would set the mode without removing the bridge, which is the silent-fallthrough case #4032 existed to prevent. It now names the call that works. The existing a_global_partitioned_topology_rejects_native_ack test passes unchanged.

What changed

• ProcessInParallelWithNativeAcks() sets NativeAck on every external slot, drops any companion local topology, and marks the topology so the rest of the pipeline can see it.
• LocalQueues() and ProcessInParallelWithNativeAcks() together are now a configuration error rather than a silently-ignored pair. Calling them in either order converges on the native-ack shape, matching how Mode() already behaves.
• AssertValidity() drops only the two rules that exist to line the companion queues up with the bridge. The subscription and external-topology rules still apply, and there is a test for that.
• GlobalPartitionedRoute skips the local shortcut. In this mode the broker delivery is the durability story, so handing a message straight to a local queue would drop it on a crash between send and handling.

Two steps from the issue that turned out to be no-ops

Worth recording, because the issue asks for both and neither is needed:

• The bridge skip needs no ListeningAgent change. The bridge is wired off Endpoint.GlobalPartitionLocalQueueUri, so not tagging it is the opt-out. There is no separate branch to add.
• The inbox-recovery skip is already free. startInboxRecoveryIfNecessary opens with if (Endpoint.Mode != EndpointMode.Durable) return;, so a native-ack slot already returns immediately. Verified rather than assumed.

That leaves EnqueueDirectlyAsync as the only ListeningAgent work in #3709's step 5, and #4040 is already in that method for #4011 — so it belongs there, not here. This PR touches no file that #4040 touches.

A transport that has not opted into NativeAck fails fast through the existing Endpoint.Mode setter, naming the endpoint type; no new validation was needed.

Scope — what is deliberately not here

The end-to-end, multi-node and slot-failover tests. No transport overrides supportsNativeAck on main yet — RabbitMQ's opt-in is in #4040 — so those cannot be written against anything real today. The NativeAckCapableEndpoint stand-in here mirrors the one #3708 already uses for the same reason.

#3709 should stay open until that coverage exists, in particular the failover test the issue calls out: killing the node owning a slot mid-handler and asserting the intra-group concurrency invariant holds across reassignment. That is the one genuine cross-node hazard in this design and nothing here proves it.

Verification

	Result
dotnet build wolverine.slnx -c Release -f net9.0	clean, 0 warnings
CoreTests	2544, 0 failed, 2 skipped — 11 new

The local-shortcut test is the one worth a reviewer's eye: it passes a null runtime, so it only survives if the shortcut is skipped outright rather than merely taking a different path.

🤖 Generated with Claude Code