Skip to content

Rebuild concurrency cap wiring + per-cell cancellation contract (closes #4710)#4878

Merged
jeremydmiller merged 1 commit into
masterfrom
feat/4710-rebuild-cap-wiring
Jul 7, 2026
Merged

Rebuild concurrency cap wiring + per-cell cancellation contract (closes #4710)#4878
jeremydmiller merged 1 commit into
masterfrom
feat/4710-rebuild-cap-wiring

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #4710. Marten-side wiring for the per-database rebuild concurrency cap (#420 / #463, epic #486 WS3), consuming JasperFx 2.23.0.

What's here

1. Resolved rebuild cap on the store (IEventStore.MaxConcurrentRebuildsPerDatabase override)

DocumentStore resolves the effective per-database rebuild cap that the projections rebuild CLI path (ProjectionHost.TryRebuildShardsAsync) reads:

  • Explicit configuration wins: StoreOptions.Projections.MaxConcurrentRebuildsPerDatabase (the new DaemonSettings knob from Address Dictionary query generation #463, inherited via ProjectionGraph — no new Marten surface needed). Zero/negative opts back into the historical unbounded fan-out.
  • Derived default: max(1, MaxPoolSize / 8) from the default database's Npgsql connection string — a 100-connection pool (Npgsql default) allows 12 concurrent rebuild cells, a 20-connection pool allows 2. Conservative by design to leave pool headroom for application traffic during rebuild windows.
  • Fallback: null (unbounded, the historical behavior) when no pool signal is reachable.

The --max-concurrent CLI flag still overrides everything per operation (that precedence lives in JasperFx.Events).

2. Descriptor population (#434 companion)

TryCreateUsage now populates EventStoreUsage.MaxConcurrentRebuildsPerDatabase with the resolved cap so monitoring tools (CritterWatch#309's rebuild dispatcher) can size their orchestration off the wire instead of falling back to a conservative 1.

3. Per-cell rebuild cancellation contract (#4710 part 2) — verified + pinned

The contract CritterWatch#309's auto-resume path depends on, now documented in rebuilding.md and pinned by per_tenant_rebuild_cancellation (TenantPartitionedEventsTests):

  • Cancelling RebuildProjectionAsync(name, tenantId, ct) mid-flight leaves the cell's mt_event_progression in a consistent state (unchanged or actual partial position — never torn).
  • A subsequent rebuild of the same (projection, tenant) cell completes successfully with no manual repair.

The mid-flight test is deterministic: the projection's ApplyAsync is TCS-gated, so cancellation provably fires while the rebuild is parked inside an apply — no sleeps, no drain loops. A pre-cancelled-token variant covers the trivial edge.

4. Docs

New rebuilding.md sections: Capping Rebuild Concurrency (knob, derived default + examples, two-layer concurrency model, rebuild-only scope vs the continuous-catch-up governors, --max-concurrent, descriptor surfacing) and Cancelling a Rebuild (the contract above).

Tests

  • rebuild_concurrency_cap_resolution (EventSourcingTests, 5): configured knob wins; non-positive disables; derived default = pool/8 (64 → 8); floors at 1 (pool 5 → 1); usage descriptor carries the effective cap.
  • per_tenant_rebuild_cancellation (TenantPartitionedEventsTests, 2): the contract tests above.
  • Full EventSourcingTests + TenantPartitionedEventsTests + CoreTests green on net9.0 (results below).

Follow-ups (not here)

🤖 Generated with Claude Code

#4710, #420/#434/#463, epic #486 WS3)

- DocumentStore resolves IEventStore.MaxConcurrentRebuildsPerDatabase:
  explicit StoreOptions.Projections knob wins, else max(1, MaxPoolSize/8)
  derived from the default database's Npgsql pool, else unbounded
- TryCreateUsage populates EventStoreUsage.MaxConcurrentRebuildsPerDatabase
  (#434 companion) for CritterWatch#309's rebuild dispatcher
- Per-tenant rebuild cancellation contract pinned with TCS-gated
  deterministic mid-flight cancellation tests (TenantPartitionedEventsTests)
- rebuilding.md: Capping Rebuild Concurrency + Cancelling a Rebuild sections
- JasperFx 2.23.0 pin

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Populate MaxConcurrentRebuildsPerDatabase + verify per-cell rebuild cancellation in TryCreateUsage

1 participant