Adopt JasperFx.Events 2.36.2: clear resolved daemons on coordinator stop, idempotent AddAsyncDaemon - #5058
Merged
Conversation
…top, idempotent AddAsyncDaemon (#5056) Closes #5055, closes #5056. At Kubernetes pod shutdown, any second Pause/Stop pass over the projection coordinator (double AddAsyncDaemon hosted-service registration, user pause + host stop, Wolverine quiesce + host stop) fanned StopAllAsync out over daemons the first pass had already disposed. Reading _cancellation.Token off the disposed CancellationTokenSource threw ObjectDisposedException, logged as "Error while trying to stop daemon agents" once per daemon on every shutdown. JasperFx.Events 2.36.2 (#574/#575) makes the daemon stop path safe after disposal and adds the abstract ClearResolvedDaemons() seam that ProjectionCoordinatorBase.StopAsync calls after disposing daemons. This is the Marten side: - ProjectionCoordinator implements ClearResolvedDaemons() by resetting the _daemons cache, so a repeated Pause/Stop has nothing to fan out over and a later ResumeAsync or daemon accessor builds fresh daemons instead of handing back disposed instances - ExplicitProjectionCoordinator.StopAsync now disposes its daemons after stopping them and clears its cache the same way - Both AddAsyncDaemon overloads skip re-registration when Marten's own coordinator is already registered, so a doubled call can't register a second IHostedService forwarding that stops the same coordinator twice Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5055, closes #5056.
The bug
At Kubernetes pod shutdown, any second Pause/Stop pass over the projection coordinator — double
AddAsyncDaemonhosted-service registration, user pause + host stop, Wolverine quiesce + host stop — fannedStopAllAsync()out over daemons the first pass had already disposed. Reading_cancellation.Tokenoff the disposedCancellationTokenSourcethrewObjectDisposedException, whichProjectionCoordinatorBase.PauseAsynclogged asError while trying to stop daemon agentsonce per daemon on every shutdown (the #5055 report).There was also a latent second bug: after
StopAsync,ResumeAsyncand the daemon accessors handed back disposed daemons from the coordinator's cache.The fix
JasperFx/jasperfx#575 (shipped in 2.36.2, tracked as #574) made the daemon stop path safe after disposal — idempotent
Dispose(),StopAllAsync()no-ops on a disposed daemon,PauseAsynclogsObjectDisposedExceptionat Debug — and added the compile-breakingprotected abstract void ClearResolvedDaemons()seam called byProjectionCoordinatorBase.StopAsyncafter it disposes the resolved daemons.This PR is the Marten side (#5056):
ProjectionCoordinatorimplementsClearResolvedDaemons()by resetting the_daemonscache under_daemonLock, so a repeated Pause/Stop has nothing to fan out over and a laterResumeAsyncor daemon accessor builds fresh daemons instead of returning disposed instancesExplicitProjectionCoordinator.StopAsyncnow disposes its daemons after stopping them and clears its cache the same way; the external manager re-acquires fresh daemons through the accessors (the same pattern the forced catch-up flow already uses)AddAsyncDaemonoverloads skip re-registration when Marten's own coordinator is already registered, so a doubled call can't register a secondIHostedServiceforwarding that makes the host stop the same coordinator twiceTests
New
Bug_5055_5056_coordinator_double_stopin DaemonTests:StopAsyncon a live coordinator logs nostop daemon agentserrors and noObjectDisposedExceptionStopAllAsync()on an already-disposed daemon no-ops (proves the 2.36.2 upstream guard is in the box)StopAsyncreturn fresh, usable daemons — not the disposed instancesAddAsyncDaemonoverloads are idempotent (single coordinator + single hosted-service registration after a doubled call)Full DaemonTests suite green on net10.0: 254 passed / 0 failed.
🤖 Generated with Claude Code