Skip to content

Fix marten#4665: CatchUpAsync dispatches on _tenantHighWater under per-tenant partitioning; bump to 2.8.2#418

Merged
jeremydmiller merged 1 commit into
mainfrom
fix/catch-up-async-tenant-partitioning-marten-4665
Jun 5, 2026
Merged

Fix marten#4665: CatchUpAsync dispatches on _tenantHighWater under per-tenant partitioning; bump to 2.8.2#418
jeremydmiller merged 1 commit into
mainfrom
fix/catch-up-async-tenant-partitioning-marten-4665

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Fixes JasperFx/marten#4665.

Bug

JasperFxAsyncDaemon.CatchUpAsync(CancellationToken) — the test-automation
catch-up path that Marten's ForceAllMartenDaemonActivityToCatchUpAsync
drives — used the store-global _highWater agent even when
_tenantHighWater was non-null.

Under per-tenant event partitioning the store-global `mt_events_sequence` is
never advanced (per-tenant `mt_events_sequence_{suffix}` values power
`mt_events.seq_id` instead). So _highWater.CheckNowAsync() leaves the
global high-water pinned at the unused sequence's last_value. Driving
catch-up off HighWaterMark() in that mode leaves every catch-up loop
stuck at zero — the test helper returned "success" while every async
projection was still behind.

The normally-running daemon already does the right thing (uses
_tenantHighWater via the polling loop and rebuildProjectionForTenant);
the test-automation CatchUpAsync was the one path still on the global
route.

Fix

When _tenantHighWater is non-null and the database implements
ICrossTenantRebuildSource (the partitioned-store contract — Marten's
MartenDatabase implements it), fan out per tenant. For each base shard:

  • Discover every tenant the projection knows about via
    ICrossTenantRebuildSource.FindRebuildTenantsAsync.
  • Activate the tenants in _tenantHighWater.PolledTenants and drive one
    vectorized poll to fetch fresh ceilings (batched — one poll round-trip
    per shard, not per tenant).
  • For each (shard, tenant), build a tenant-scoped agent
    (asyncShard with { Name = asyncShard.Name.ForTenant(tenantId) }) and
    catch it up to that tenant's ceiling.

Mirrors the rebuildProjectionForTenant ceiling-lookup pattern that
already existed for per-tenant rebuilds.

Single-tenant stores and non-partitioned multi-tenant stores keep the
byte-for-byte global path.
The dispatch is gated on
_tenantHighWater != null && Database is ICrossTenantRebuildSource — no
other behavior changes.

Version

2.8.0 → 2.8.2 (skipping 2.8.1 to avoid clashing with any local
prerelease lines).

Marten-side regression

JasperFx/marten#4673 ships
the failing reproduction (currently Skip'd to keep CI from hanging).
After this lands on NuGet and Marten bumps Directory.Packages.props to
2.8.2, the Marten regression test gets unskipped.

Verification

Step Result
dotnet build src/JasperFx.Events/JasperFx.Events.csproj -f net9.0 clean
dotnet test src/EventStoreTests -f net9.0 72 / 72 ✅
dotnet test src/EventTests -f net9.0 351 / 351 ✅

🤖 Generated with Claude Code

…r-tenant partitioning; bump to 2.8.2

JasperFxAsyncDaemon.CatchUpAsync(CancellationToken) — the test-automation
catch-up path that ForceAllMartenDaemonActivityToCatchUpAsync drives —
used the store-global _highWater path even when _tenantHighWater was
non-null. Under per-tenant event partitioning the store-global
mt_events_sequence is never advanced (per-tenant mt_events_sequence_{suffix}
values power mt_events.seq_id), so _highWater.CheckNowAsync() leaves the
global high-water pinned at the unused sequence's last_value. Driving
catch-up off HighWaterMark() in that mode leaves every catch-up loop
stuck at zero — the helper returned "success" while every async projection
was still behind.

Fix: when _tenantHighWater is non-null AND the database implements
ICrossTenantRebuildSource (the partitioned-store contract — Marten's
MartenDatabase implements it), fan out per tenant. For each base shard:

* Discover every tenant the projection knows about via
  ICrossTenantRebuildSource.FindRebuildTenantsAsync.
* Activate the tenants in _tenantHighWater.PolledTenants and drive one
  vectorized poll to fetch fresh ceilings.
* For each (shard, tenant), build a tenant-scoped agent
  (asyncShard with { Name = asyncShard.Name.ForTenant(tenantId) }) and
  catch it up to that tenant's ceiling.

Mirrors the rebuildProjectionForTenant ceiling-lookup pattern that already
existed for per-tenant rebuilds. Single-tenant stores and non-partitioned
multi-tenant stores keep the byte-for-byte global path.

Marten side: src/TenantPartitionedEventsTests/Regressions/Bug_4665_catch_up_uses_global_high_water.cs
(JasperFx/marten#4673) is the failing reproduction shipped as Skip pin;
when this lands on NuGet and Marten bumps Directory.Packages.props to
2.8.2, the test gets unskipped.

Version bump: 2.8.0 → 2.8.2 (skipping 2.8.1 to avoid clashing with any
local prerelease lines).

Verified locally:
* dotnet build src/JasperFx.Events/JasperFx.Events.csproj -f net9.0 — clean.
* dotnet test src/EventStoreTests -f net9.0 — 72/72 ✅.
* dotnet test src/EventTests -f net9.0 — 351/351 ✅.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jeremydmiller jeremydmiller force-pushed the fix/catch-up-async-tenant-partitioning-marten-4665 branch from 06a130d to 92a9a16 Compare June 5, 2026 19:19
@jeremydmiller jeremydmiller merged commit e8a08ac into main Jun 5, 2026
1 check passed
@jeremydmiller jeremydmiller deleted the fix/catch-up-async-tenant-partitioning-marten-4665 branch June 5, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant