feat(#335): per-tenant managed partitioning parity with Marten (documents + streams) - #338
Merged
Merged
Conversation
Weasel 9.18.0 ships weasel#362 (PR weasel#374) — ManagedTenantPartitions parity with the PostgreSQL ManagedListPartitions: TenantDropBehavior (RetainData/DeleteData purge-before-merge), AllowOrdinalSharing + explicit-ordinal add overloads (tenant bucketing under the 15,000-partition ceiling), MigrateAllTablesAsync new-table back-fill, and TenantPartitionAddResult / TablePartitionStatus batch status reporting. The prerequisite for polecat#335. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ents + streams)
Brings Polecat's managed per-tenant partitioning to full Marten
equivalence on Weasel 9.18.0:
- pc_streams now partitions alongside pc_events under
UseTenantPartitionedEvents (mt_streams parity): tenant_ordinal joins
the PK after (tenant_id, id) so readers keep their prefix seek, the
stream INSERT stamps the planner-resolved ordinal, and the version
UPDATE partition-eliminates on it.
- Conjoined document tables can be tenant-partitioned: the DocumentTable
"single-tenant only" restriction is lifted for the managed strategy —
tables gain a tenant_ordinal PK column driven by the one
pc_tenant_partitions registry per database. Every write path (MERGE
upsert/insert/overwrite, update-only, version-checked bulk) resolves
the ordinal SERVER-SIDE by joining the registry into the MERGE source,
so parameter slots are unchanged and cross-process ordinal state can
never mis-route rows; the ON clause carries tenant_ordinal for
partition elimination.
- Policy APIs: Policies.AllDocumentsAreMultiTenantedWithPartitioning()
(forces conjoined tenancy) and
Policies.PartitionMultiTenantedDocumentsUsingPolecatManagement(),
with a per-type ForDocument<T>(p => p.DisableTenantPartitioning)
escape hatch; the daemon's DeadLetterEvent document is always
excluded (Marten parity).
- Runtime tenant onboarding:
store.Advanced.AddPolecatManagedTenantsAsync (batch add with
TablePartitionStatus[] reporting + explicit-ordinal bucketing
overload) and RemovePolecatManagedTenantsAsync with
TenantDropBehavior — DeleteData purges the tenants' rows before
MERGE RANGE (weasel#362 item 1), fully-released ordinals drop their
pc_events_sequence_{ordinal}, and in-process caches are evicted so a
removed tenant can be re-onboarded.
- Tenants are provisioned lazily on first write everywhere: the session
flush pipeline, the daemon's projection batch, and both bulk-insert
surfaces resolve ordinals through the new shared
TenantPartitionOrdinalRegistry (TenantEventSequenceRegistry now
composes it, so events/streams/documents share one tenant -> ordinal
cache). Table creation hydrates the manager first so tables created
after tenants exist bake the full boundary set.
No reserved default-tenant partition is needed: Polecat has no global
projections, and integer ordinals sidestep Marten's suffix-identifier
constraint entirely.
15 new integration tests (streams schema/placement/update path; document
schema shape, ordinal stamping, ForTenant, bulk, opt-out, config guards,
add/remove with both drop behaviors, shared-registry coherence across
pc_events/pc_streams/documents); docs updated.
Closes #335.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #335. Depends on (and bumps to) Weasel 9.18.0, which ships the JasperFx/weasel#362
ManagedTenantPartitionsparity work (JasperFx/weasel#374).What's here
Weasel 9.17.0 → 9.18.0 — picks up
TenantDropBehavior(RetainData/DeleteData purge-before-merge),AllowOrdinalSharing+ explicit-ordinal bucketing overloads,MigrateAllTablesAsyncback-fill, andTenantPartitionAddResult/TablePartitionStatus[]batch status reporting.pc_streamspartitions alongsidepc_events(issue scope 2,mt_streamsparity): underUseTenantPartitionedEvents,tenant_ordinaljoins the streams PK after(tenant_id, id)so every existing reader keeps its prefix seek; the stream INSERT stamps the planner-resolved ordinal and the version UPDATE partition-eliminates on it.Tenant-partitioned document tables (scope 1): the
DocumentTable"single-tenant only" throw is lifted for the managed strategy. Conjoined document tables gain atenant_ordinal intPK column on the store's sharedManagedTenantPartitions— onepc_tenant_partitionsregistry per database, shared with the event store. The ordinal is resolved server-side in every write statement by joining the registry into the MERGEUSINGsource ((VALUES ...) v LEFT JOIN pc_tenant_partitions tp ON tp.tenant_id = v.tenant_id), which keeps every positional parameter slot unchanged across the upsert/insert/overwrite/update-only/version-checked-bulk shapes and makes cross-process ordinal state incapable of mis-routing rows. TheONclause carriestenant_ordinalfor partition elimination (Marten #4223 analogue).Policy APIs (scope 3):
Policies.AllDocumentsAreMultiTenantedWithPartitioning()(forces conjoined tenancy) andPolicies.PartitionMultiTenantedDocumentsUsingPolecatManagement(), plus a per-typeForDocument<T>(p => p.DisableTenantPartitioning = true)escape hatch. The daemon'sDeadLetterEventdocument is always excluded (Marten parity).Runtime tenant onboarding (scopes 4+5):
store.Advanced.AddPolecatManagedTenantsAsync(ct, params tenantIds)returns per-tableTablePartitionStatus[], with an explicit-ordinal dictionary overload for tenant bucketing;RemovePolecatManagedTenantsAsynctakesTenantDropBehavior—DeleteDataphysically purges the tenants' rows from every managed table beforeMERGE RANGE(the weasel#362 item-1 semantics), fully-released ordinals drop theirpc_events_sequence_{ordinal}, and in-process caches evict so a removed tenant can be re-onboarded fresh.Lazy provisioning everywhere: a new shared
TenantPartitionOrdinalRegistry(composed byTenantEventSequenceRegistry, so events/streams/documents share one tenant → ordinal cache) is resolved by the session flush pipeline, the daemon's projection batch, and both bulk-insert surfaces before operations execute. Table creation hydrates the manager first, so a table created after tenants exist bakes the full boundary set into itsCREATE.No reserved default-tenant partition: Polecat has no global projections, and integer ordinals sidestep Marten's
__default__suffix-identifier constraint entirely — documented rather than ported.Testing
15 new integration tests in the serialized
tenant-partitioningcollection (streams schema/physical placement/append-update path; document schema shape + ordinal stamping via$PARTITION, ForTenant override, bulk insert, per-type opt-out, config guards, add/remove with both drop behaviors incl. re-onboarding, and events+streams+documents sharing one registry/ordinal). Full net10 suite locally: 1480 passed / 0 failed on a fresh SQL Server 2025 container.Follow-up filed: JasperFx/wolverine#3496 (EF Core integration adopting the same Weasel 9.18.0 surface).
🤖 Generated with Claude Code