feat(durability): AlwaysMakeScheduledMessagesDurable policy#2657
Merged
jeremydmiller merged 1 commit intomainfrom May 1, 2026
Merged
feat(durability): AlwaysMakeScheduledMessagesDurable policy#2657jeremydmiller merged 1 commit intomainfrom
jeremydmiller merged 1 commit intomainfrom
Conversation
…uffered local queues Adds a new policy on `WolverineOptions.Policies` that promotes scheduled-for-later envelopes destined for non-durable `BufferedLocalQueue` instances onto the durable `IMessageStore.Inbox` instead of the in-process `IScheduledJobProcessor`. Without this opt-in, those envelopes live only in memory and are silently lost when the process restarts. Scope is narrower than I initially scoped while implementing — the rest of the matrix already provides durability: - Native broker scheduling (Azure Service Bus / Pulsar / Redis / Pub/Sub): persisted server-side by the broker. - Non-native broker senders (RabbitMQ / SQS / Kafka): the routing layer (`MessageRoute.WriteEnvelope`) already swaps scheduled envelopes onto the `local://durable` system queue, which writes to the message store inbox. - Local queues with `UseDurableInbox()`: already write to the message store via `DurableLocalQueue`. The unique gap this policy plugs is the default `BufferedInMemory` local queue case. The hook lives in `BufferedLocalQueue.EnqueueOutgoingAsync`: when the flag is set, the storage isn't `NullMessageStore`, and the envelope is scheduled, it goes to `Storage.Inbox.RescheduleExistingEnvelopeForRetryAsync` (the misnamed upsert path used for new scheduled envelopes — same as `MessageContext.flushScheduledMessagesAsync`) instead of the in-process scheduler. Recovery-path enqueues (`IListenerCircuit.EnqueueDirectlyAsync`) bypass the hook because those envelopes already came from the message store. Startup wiring emits a warning when the flag is on but `Storage is NullMessageStore` so a misconfiguration surfaces rather than silently degrading. Tests cover the two cases the user explicitly asked for: - `PostgresqlTests/scheduled_messages_use_message_store_when_AlwaysMakeScheduledMessagesDurable_is_set`: buffered local queue + Postgres. With policy: scheduled rows land in the inbox. Without policy: they don't (in-process scheduler). Tested with both a plain message and a `TimeoutMessage` subtype to confirm the policy isn't gated on message base class. - `Wolverine.RabbitMQ.Tests/scheduled_messages_use_message_store_when_AlwaysMakeScheduledMessagesDurable_is_set`: non-durable RabbitMQ + Postgres. Both with-and-without policy: rows land in the inbox via the existing routing-layer swap to `local://durable`. Locks down the invariant so a future routing refactor can't silently break durability for scheduled sends to non-native broker transports. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
WolverineOptions.Policies.AlwaysMakeScheduledMessagesDurable(). When set, scheduled-for-later envelopes destined for non-durableBufferedLocalQueueinstances route toIMessageStore.Inboxinstead of the in-processIScheduledJobProcessor, so they survive process restarts.Architectural note (scope ended up narrower than initially scoped)
The original intent was to cover all four cases where scheduled messages might be lost — local queues (in-process scheduler) and non-durable broker senders (in-memory fallback). On implementing it became clear the routing layer already covers most of the matrix:
MessageRoute.WriteEnvelopeswaps ontolocal://durable→ already in MessageStoreUseDurableInbox()DurableLocalQueueBufferedInMemory(default)IScheduledJobProcessor— lost on restartSo the unique gap the policy plugs is the bottom row. The hook lives in
BufferedLocalQueue.EnqueueOutgoingAsync. I removed the broaderMessageContext/MessageBushooks I'd initially added as dead code — they would only fire if a future change broke the routing-layer swap, in which case the right fix is repairing the swap rather than carrying a defensive duplicate.Tests
Two new test files, covering the two cases in the original ask:
PostgresqlTests/scheduled_messages_use_message_store_when_AlwaysMakeScheduledMessagesDurable_is_set— buffered local queue + Postgres. With policy: 2 scheduled rows in the inbox. Without policy: 0 (in-process scheduler is the destination). Tested with both a plain message and aTimeoutMessagesubtype.Wolverine.RabbitMQ.Tests/scheduled_messages_use_message_store_when_AlwaysMakeScheduledMessagesDurable_is_set— non-durable RabbitMQ + Postgres. Both with-and-without policy: 2 scheduled rows in the inbox via the existinglocal://durableswap. Locks down the routing-layer invariant so a future refactor can't silently regress durability for non-native broker scheduling.Startup wiring also emits a warning when the flag is on but
Storage is NullMessageStore, so a misconfiguration surfaces rather than silently degrading.Test plan
dotnet test src/Persistence/PostgresqlTests --filter "FullyQualifiedName~AlwaysMakeScheduledMessagesDurable"— 2/2dotnet test src/Transports/RabbitMQ/Wolverine.RabbitMQ.Tests --filter "FullyQualifiedName~AlwaysMakeScheduledMessagesDurable"— 2/2dotnet test src/Testing/CoreTests --filter "FullyQualifiedName~Scheduled | FullyQualifiedName~Routing"— no regressions (53 pass locally)🤖 Generated with Claude Code