fix(blocks): Post no longer silently drops items on a full bounded channel#485
Merged
Conversation
…annel Block<T>.Post used a non-blocking TryWrite against a channel bounded at 10,000 and silently dropped every item once the channel filled -- only a Debug.WriteLine marked the loss. Callers that post from outside the reader (Wolverine's buffered local queues, Marten's ProjectionUpdateBatch) lost data under load with no error. Reported as wolverine GH-3287, where a handler cascading 20k messages to a local queue only delivered ~10k. - Post now block-waits for capacity via WriteAsync (BoundedChannelFullMode.Wait) instead of dropping, so writers are back-pressured rather than losing items. - Add a configurable boundedCapacity plus an Unbounded option for blocks that must never block or drop on write. - RetryBlock uses an unbounded internal block: executeAsync re-enqueues failed items onto its own block from within its processing action, which would deadlock against back pressure on a full bounded channel. Bumps to 2.19.1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Block<T>.Postwrote to a channel bounded at 10,000 using a non-blockingTryWrite, and silently dropped every item once the channel filled — the only trace was aDebug.WriteLine. Any caller that posts faster than the reader drains loses data with no error surfaced.This is the root cause of wolverine GH-3287: a handler cascading 20,000 messages to a buffered local queue only delivered ~10,000. It also silently affects Marten's
ProjectionUpdateBatch(a projection producing >10k storage operations in one batch would drop the overflow).Fix
Postback-pressures instead of dropping. On a full bounded channel it now block-waits for capacity viaWriteAsync(BoundedChannelFullMode.Wait). Safe for the common case where the poster is not the block's own reader.Unboundedoption. NewBlock(int parallelCount, int boundedCapacity, action)ctor; passBlock<T>.Unboundedfor blocks that must never block or drop. The parameterless/(parallelCount, action)ctors keep theDefaultBoundedCapacity(10,000) — no behavior change for existing callers except that overflow now blocks rather than drops.RetryBlockuses an unbounded internal block.executeAsyncre-enqueues failed items onto its own block from within its processing action; against a full bounded channel with back pressure that self-re-post would deadlock, so retries must never block on write.Downstream (Wolverine)
Wolverine's buffered local queue will opt into
Block<Envelope>.Unbounded(a local queue is routinely a handler's own cascade target, so bounded back pressure could deadlock the single reader). That change ships in the paired Wolverine PR once 2.19.1 is published.Tests
InMemoryQueueTests.synchronous_Post_does_not_drop_items_when_the_bounded_channel_fills— holds the reader closed until the channel saturates and the producer is blocked inPost, proving the overflow item is back-pressured, not dropped; then drains all 25,000.InMemoryQueueTests.unbounded_block_never_blocks_or_drops_on_Post.CoreTestssuite green (453 passed), including existingRetryBlocktests.Bumps to 2.19.1.
🤖 Generated with Claude Code