Skip to content

Pin subscription rewind under per-tenant partitioning end to end - #5719

Merged
jeremydmiller merged 2 commits into
JasperFx:masterfrom
lahma:fix/rewind-under-partitioning-e2e
Oct 23, 2026
Merged

jeremydmiller merged 2 commits into
JasperFx:masterfrom
lahma:fix/rewind-under-partitioning-e2e

Conversation

@lahma

@lahma lahma commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Why

#1025 (JasperFx 2.91.0) rewinds a subscription per tenant under UseTenantPartitionedEvents. It calls RewindSubscriptionProgressAsync with each tenant shard's own identity, which Marten accepts since #5700. #1025's tests run the daemon against a substituted store, and its body planned Marten end-to-end tests for the bump. #5713 did the bump and turned rewind_subscription_per_tenant_pin into a test of the new behaviour (1ee3615). It pins three things: a per-tenant rewind of a projection to the beginning, that the sibling tenant's progression row is untouched, and that the tenant-less rewind refuses sequenceFloor: 1. This PR adds the rest of the end-to-end coverage, against a live store, with a raw ISubscription.

Assertion On master today Here
A tenant-less rewind to 0 restarts one agent per tenant, never a store-global RewindProbe:All Only that it doesn't throw, with no agent running (tenantless_RewindSubscriptionAsync_does_NOT_throw_under_partitioning). JasperFx covers it against a substitute. Live
...and each tenant replays only its own events, ending at its own height (5 and 3) None Yes
A per-tenant rewind to a floor above 0 replays only that tenant's events above the floor Floor 0 for a projection (#5713) Floor 2: events 3 to 5 only
...and deletes only that tenant's dead letters at or above the floor Only through RewindSubscriptionProgressAsync called directly (#5700) Through RewindSubscriptionAsync(name, tenantId, …)
...and leaves the other tenant's agent instance running JasperFx, against a substitute Live: same instance, Running
...and leaves the other tenant's progression row #5713 Yes
A tenant-less rewind to a sequence number is refused #5713 (floor 1) Floor 2
...and changes no agent, progression row or dead letter in either tenant JasperFx, against a substitute (the store is never called) Live

Test

New file: TenantPartitionedEventsTests/Daemon/rewind_subscription_under_partitioning.cs. It uses PartitionedStoreContext, a recording subscription and two tenants at heights 5 and 3:

  • rewinding_every_tenant_to_the_beginning_replays_each_tenant_through_its_own_agent
  • rewinding_one_tenant_moves_only_that_tenant
  • rewinding_every_tenant_to_a_sequence_number_is_refused_and_changes_nothing

No production code changes.

Red and green (net10.0, PostgreSQL 17)

Red on 8065f4467, master's commit before #5700 merged (JasperFx 2.90.1): 3 of 3 fail.

rewinding_every_tenant_to_the_beginning_replays_each_tenant_through_its_own_agent
  runningIdentities(daemon) should be ["RewindProbe:All:t_769fb6dba540", "RewindProbe:All:t_6e70499012dd"] but was ["RewindProbe:All"]
rewinding_every_tenant_to_a_sequence_number_is_refused_and_changes_nothing
  await deadLetterSequencesAsync(beta, token) should be [2L, 3L] but was []      <- the other tenant's dead letters were deleted
rewinding_one_tenant_moves_only_that_tenant
  System.NotSupportedException : Per-tenant RewindSubscriptionAsync is not implemented on this IProjectionDaemon.

On today's master, with #5700's source change reverted locally, the first two fail with Unknown subscription name 'RewindProbe:All:t_…'. With only #5700's TenantId predicate removed from the dead-letter delete, rewinding_one_tenant_moves_only_that_tenant fails with deadLetterSequencesAsync(beta) should be [2L, 3L] but was [].

Green on master 7045d09bb: 3 of 3, and 10 of 10 consecutive runs. TenantPartitionedEventsTests.Daemon: 41 of 41.

Searched open and closed issues and PRs for "rewind subscription tenant", "RewindSubscriptionAsync partitioned" and "rewind dead letters tenant". Only #5700, #5713, #4856, #4617 and #5294 come up, and none adds these tests.

🤖 Generated with Claude Code

lahma and others added 2 commits October 20, 2026 11:19
JasperFx 2.91.0 (JasperFx#1025) rewinds a subscription per tenant under
UseTenantPartitionedEvents, through RewindSubscriptionProgressAsync with each tenant
shard's own identity, which Marten accepts since JasperFx#5700. The daemon logic is pinned in
JasperFx against a substituted store, and rewind_subscription_per_tenant_pin (JasperFx#5713)
pins a per-tenant rewind of a projection to the beginning. Nothing in Marten pinned the
rest against a live store.

Three facts, with a raw ISubscription and two tenants at heights 5 and 3:

- Rewinding every tenant to the beginning restarts each tenant's own agent, never a
  store-global one, and each replays only its own tenant's events.
- Rewinding one tenant to sequence floor 2 replays only that tenant's events above it,
  deletes only that tenant's dead letters at or above it, and leaves the other tenant's
  agent instance running with its progression and dead letters untouched.
- Rewinding every tenant to a sequence number is refused, and changes no agent, no
  progression row and no dead letter.

All three fail on 8065f44, the master commit before JasperFx#5700 merged (JasperFx 2.90.1),
and pass on master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A subscription sees a page in ProcessEventsAsync, but the page's progression
row is only committed with the batch afterwards. Asserting progress the moment
the events were seen raced that commit, most visibly right after a rewind,
where the row still holds the floor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants