GH-3709: send inline from NativeAck endpoints to close the interceptor loss window

[jeremydmiller]

Addresses item 4 of #3709 — see the analysis comment. Does not close the issue; the multi-node and failover tests remain.

The bug

GlobalPartitionedInterceptor.TryReRouteAsync re-publishes a message and then acknowledges the source delivery:

await bus.PublishAsync(envelope.Message!, options);
await listener.CompleteAsync(envelope);

#4040 mapped EndpointMode.NativeAck to BufferedSendingAgent, reasoning that the mode means "no outbox" — which is what BufferedInMemory means. That is correct for ordinary sends and wrong for this particular sequence. BufferedSendingAgent.storeAndForwardAsync posts to an in-memory Block, so the source delivery could be settled while the only copy of the message was sitting in this process's memory. Crash in between and it is gone, with no redelivery available because the source was already acked.

Under the durable topology this hop was safe for the reason #3709 item 4 predicted: the re-publish landed in the durable outbox before the source ack. #4041 made native-ack global partitioning reachable, which made this live on main.

The fix

NativeAck now maps to InlineSendingAgent, which posts to a RetryBlock that runs the send rather than queueing it.

The reasoning for doing it at the mode level rather than special-casing the interceptor: NativeAck is a listening optimization. Nobody selects it for its sending characteristics, so the outgoing side should take the safe option rather than the fast one. A narrower special-case (Mode == NativeAck && UsedInShardedTopology) was considered and rejected as more machinery for no benefit.

Two limits, stated rather than glossed

This narrows the window rather than closing it. Wolverine publishes with RabbitMQ publisher confirms disabled by default, so an inline send awaits the frame being written, not the broker acknowledging it. Enabling confirms for these endpoints is the remaining step and is not in this PR.

The window was never deterministic. Block<T>.PostAsync runs its handler inline when the block is idle, so the buffered agent happened to deliver synchronously for a single envelope. The gap opens only when the block already has queued work or a send is retrying — making this a load-dependent bug that works fine in tests and loses messages in production. My original write-up on #3709 implied it was deterministic; it is not, and that makes it harder to catch, not easier.

On the tests

There is deliberately no behavioral "the send reached the transport before the call returned" test. One was written. It passes identically under both mappings, for the reason above, and was deleted rather than kept as decoration — a test that cannot fail is worse than no test, because it reads like coverage.

What is here is the agent-type assertions, plus one guarding that BufferedInMemory still gets the buffered agent so the change is not over-applied. Both were red-baselined: with the NativeAck arm reverted to BufferedSendingAgent, a_native_ack_endpoint_gets_the_inline_sending_agent fails and the buffered one still passes.

Note the tests use a purpose-built Endpoint rather than StubEndpoint. StubEndpoint assigns Agent = this in its constructor and is its own ISendingAgent, so it bypasses buildSendingAgent entirely — the first version of these tests used it and passed under both mappings for that reason too.

Verification

• dotnet build wolverine.slnx -c Release -f net9.0 — clean, 0 warnings, 0 errors
• CoreTests — 2552 total, 0 failed, 2 skipped

🤖 Generated with Claude Code