Stop daemon-internal cancellations from leaking into CatchUpAsync (closes #284)#285
Merged
Merged
Conversation
…oses #284) GroupedProjectionExecution.processRangeAsync ignored the caller token and used its private _cancellation CTS for batch builds. When that CTS fired mid-batch (StopAllAsync / HardStopAsync / Dispose during a rebuild), the resulting OperationCanceledException (or Npgsql 57014 wrapper) flowed into the generic catch and was reported via ReportCriticalFailureAsync, which Recorder captured onto a ShardState and CatchUpAsync re-threw as an AggregateException. Marten's ForceAllMartenDaemonActivityToCatchUpAsync test helper then surfaced this as a spurious "exceptions should be empty" failure even though no caller cancellation ever happened. Two-layer fix: 1. Add catch guards in GroupedProjectionExecution.processRangeAsync and groupEventRangeAsync that short-circuit when _cancellation.IsCancellationRequested is true, matching the existing pattern in applyBatchOperationsToDatabaseAsync and SubscriptionExecutionBase.executeRange. 2. Defense in depth in JasperFxAsyncDaemon.CatchUpAsync: drop OCE-shaped exceptions (including AggregateExceptions whose inners are all OCE) from the aggregated set when the inbound cancellation token was never cancelled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
4 tasks
This was referenced May 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GroupedProjectionExecution.processRangeAsync(andgroupEventRangeAsync) now short-circuit OCE / wrapped-OCE in their generic catch when the shard's private_cancellationCTS has fired, instead of pushing the noise throughReportCriticalFailureAsync. Mirrors the existing guards inapplyBatchOperationsToDatabaseAsyncandSubscriptionExecutionBase.executeRange.JasperFxAsyncDaemon.CatchUpAsyncfilters cancellation-shaped exceptions out of the aggregatedrecorder.Statesset when the inbound cancellation token was never cancelled (defense in depth, covers theAggregateException-of-OCEshape from the issue trace).Why
Per #284 (JasperFx-side follow-up to JasperFx/marten#4462): Marten's
ForceAllMartenDaemonActivityToCatchUpAsynctest helper callsStopAllAsync()immediately beforeCatchUpAsync. Daemon-internal cancellation of an in-flight batch build (Npgsql 57014 surfaced throughFetchProjectionStorageAsync) was being recorded ontoShardState.Exception, aggregated atJasperFxAsyncDaemon.cs:716, and re-thrown — making the helper surface benign internal-cancellation as a misleading "exceptions should be empty but had 1 item" assertion. The user-supplied CT never fired, so this was always internal-lifecycle noise.Matches fix-path (2) + (3) from the issue's suggested fixes.
Test plan
dotnet build src/JasperFx.Events/JasperFx.Events.csproj— clean (216 pre-existing nullability warnings)dotnet test src/EventTests/EventTests.csproj— 270/270 pass on net9.0 and net10.0dotnet test src/EventStoreTests/EventStoreTests.csproj— 72/72 net9.0; 70/72 net10.0 (2 pre-existingRecentlyUsedCacheTestsflakes unrelated to daemon code; pass in isolation on bothmainand this branch)Bug_4441_force_catch_up_with_outbox.force_catch_up_invokes_message_batch_lifecycle_with_custom_outbox(currently pointed at marten#4462)🤖 Generated with Claude Code