Fix unbounded retry loop on durable receiver shutdown (GH-2671)

[jeremydmiller]

Closes #2671.

Diagnosis

DurableReceiver.executeWithRetriesAsync was an unbounded while (true) loop that:

1. Never gave up — no retry cap
2. Ignored cancellation — neither the loop guard nor the Task.Delay honoured the durability cancellation token
3. Always logged at Error level — including expected failures during host teardown

On shutdown, DrainAsync calls _inbox.ReleaseIncomingAsync(...) through this helper. When the Npgsql DbDataSource has already been disposed by DI ordering, every retry hammers a dead socket and emits a SocketException at Error level. The user reported the flood started after upgrading 5.22 → 5.27 — PR #2391's shutdown reordering made DrainAsync reliably reach this code path on a torn-down data source, exposing the latent bug.

Fix

Three changes to executeWithRetriesAsync:

1. Bounded retries — cap at MaxReleaseRetries = 5. After the budget is exhausted, log a single Error and return rather than spinning forever. Any owned envelopes left as owner_id = node_id are reclaimed by DurabilityAgent's recovery polling on the next live node, so silently giving up is functionally safe for this best-effort cleanup path.

2. Cancellation-aware exit — when _settings.Cancellation is signalled (which happens via DurabilitySettings.Cancel() during shutdown), bail out on the first failure. The cancellation token is also passed to Task.Delay so the backoff doesn't sleep through the entire shutdown sequence.

3. Demoted log level on cancelled exit — these failures are expected during teardown and don't deserve Error-level noise. Outside cancellation, the existing Error log is preserved with attempt counters added for diagnostics.

Test plan

New durable_receiver_release_incoming_during_shutdown test class:

• drain_terminates_within_seconds_when_inbox_release_throws_repeatedly asserts DrainAsync finishes inside a 5-second budget when the inbox throws SocketException on every call — the pre-fix code loops forever here.
• drain_exits_immediately_when_cancellation_is_signalled cancels DurabilitySettings.Cancel() first and asserts the drain exits inside one second.
• Full Runtime.WorkerQueues suite green (25/25) on net9.0 — existing receiver tests are unaffected.

🤖 Generated with Claude Code