Rebuild concurrency cap wiring + per-cell cancellation contract (closes #4710)#4878
Merged
Merged
Conversation
#4710, #420/#434/#463, epic #486 WS3) - DocumentStore resolves IEventStore.MaxConcurrentRebuildsPerDatabase: explicit StoreOptions.Projections knob wins, else max(1, MaxPoolSize/8) derived from the default database's Npgsql pool, else unbounded - TryCreateUsage populates EventStoreUsage.MaxConcurrentRebuildsPerDatabase (#434 companion) for CritterWatch#309's rebuild dispatcher - Per-tenant rebuild cancellation contract pinned with TCS-gated deterministic mid-flight cancellation tests (TenantPartitionedEventsTests) - rebuilding.md: Capping Rebuild Concurrency + Cancelling a Rebuild sections - JasperFx 2.23.0 pin Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4710. Marten-side wiring for the per-database rebuild concurrency cap (#420 / #463, epic #486 WS3), consuming JasperFx 2.23.0.
What's here
1. Resolved rebuild cap on the store (
IEventStore.MaxConcurrentRebuildsPerDatabaseoverride)DocumentStoreresolves the effective per-database rebuild cap that theprojections rebuildCLI path (ProjectionHost.TryRebuildShardsAsync) reads:StoreOptions.Projections.MaxConcurrentRebuildsPerDatabase(the newDaemonSettingsknob from Address Dictionary query generation #463, inherited viaProjectionGraph— no new Marten surface needed). Zero/negative opts back into the historical unbounded fan-out.max(1, MaxPoolSize / 8)from the default database's Npgsql connection string — a 100-connection pool (Npgsql default) allows 12 concurrent rebuild cells, a 20-connection pool allows 2. Conservative by design to leave pool headroom for application traffic during rebuild windows.null(unbounded, the historical behavior) when no pool signal is reachable.The
--max-concurrentCLI flag still overrides everything per operation (that precedence lives in JasperFx.Events).2. Descriptor population (#434 companion)
TryCreateUsagenow populatesEventStoreUsage.MaxConcurrentRebuildsPerDatabasewith the resolved cap so monitoring tools (CritterWatch#309's rebuild dispatcher) can size their orchestration off the wire instead of falling back to a conservative 1.3. Per-cell rebuild cancellation contract (#4710 part 2) — verified + pinned
The contract CritterWatch#309's auto-resume path depends on, now documented in
rebuilding.mdand pinned byper_tenant_rebuild_cancellation(TenantPartitionedEventsTests):RebuildProjectionAsync(name, tenantId, ct)mid-flight leaves the cell'smt_event_progressionin a consistent state (unchanged or actual partial position — never torn).The mid-flight test is deterministic: the projection's
ApplyAsyncis TCS-gated, so cancellation provably fires while the rebuild is parked inside an apply — no sleeps, no drain loops. A pre-cancelled-token variant covers the trivial edge.4. Docs
New
rebuilding.mdsections: Capping Rebuild Concurrency (knob, derived default + examples, two-layer concurrency model, rebuild-only scope vs the continuous-catch-up governors,--max-concurrent, descriptor surfacing) and Cancelling a Rebuild (the contract above).Tests
rebuild_concurrency_cap_resolution(EventSourcingTests, 5): configured knob wins; non-positive disables; derived default = pool/8 (64 → 8); floors at 1 (pool 5 → 1); usage descriptor carries the effective cap.per_tenant_rebuild_cancellation(TenantPartitionedEventsTests, 2): the contract tests above.Follow-ups (not here)
DocumentStore(Patching API should allow adding and removing properties #434 companion list).JasperFxAsyncDaemon(tracked on New mechanism for adding arbitrary database operations to the unit of work batch in DocumentSession #420's follow-up list).🤖 Generated with Claude Code