Skip to content

Make managed tenant partition bucketing actually share a partition (#391) - #392

Merged
jeremydmiller merged 2 commits into
masterfrom
weasel-391/managed-partition-bucketing
Jul 28, 2026
Merged

Make managed tenant partition bucketing actually share a partition (#391)#392
jeremydmiller merged 2 commits into
masterfrom
weasel-391/managed-partition-bucketing

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #391.

Both managed-partition strategies advertise multi-tenant bucketing — several small tenants sharing one
physical partition to stay clear of per-table partition ceilings — but neither could express it end to end.
Both halves reproduced against real PostgreSQL and SQL Server before fixing.

PostgreSQL — ManagedListPartitions

AddPartitionToAllTables emitted one single-value ListPartition per value, so two values sharing a suffix
produced two CREATE TABLE IF NOT EXISTS statements against the same partition table name. The first created
it with only its own value; the second was silently swallowed. The partition never accepted the second value,
and that tenant's first write died with 23514 no partition of relation found for row.

The additive path now groups by sanitized suffix — that is what names the physical table — and carries the
bucket's full membership, not just the values supplied in the current call. Since CREATE TABLE IF NOT EXISTS
cannot alter an existing bound, widening a partition that is already there is DETACH + re-ATTACH inside
one transaction; a failure between the two would orphan the partition and leave every tenant in the bucket
unroutable.

Also fixed here: co-tenant data loss on drop

DropPartitionFromAllTablesForValue resolved a value to its suffix and then dropped by suffix — deleting
every registry row in the bucket and DROPping the shared partition table. Removing one small tenant therefore
silently destroyed its co-tenants' rows. It now narrows the bucket instead: only the departing value's rows are
deleted, the partition is re-bound to the remaining members, and the partition is released only with the last
member. Filed as JasperFx/wolverine#3686.

SQL Server — ManagedTenantPartitions

Bucketing worked inside a single batch, because the caller supplied the shared ordinal itself. Across calls —
the natural tenant-onboarding shape, and the one the Wolverine docs showed — it silently did not. The registry
persisted only tenant_id -> ordinal, so a brand-new tenant had no way to discover which ordinal its bucket
already owned; each call allocated a fresh one and the tenants never actually shared. The 15,000-partition
ceiling bucketing exists to dodge was not mitigated at all.

The registry gains a nullable bucket column, plus a bucket-aware AddPartitionsToAllTables overload taking
tenant -> bucket. Deliberately a column rather than the pseudo-tenant row considered in #391: a row
would keep the ordinal referenced forever, defeating release-on-last-member and therefore
TenantDropBehavior.DeleteData. A bucket is forgotten once its last member is dropped.

New public surface: the IReadOnlyDictionary<string, string?> overload of AddPartitionsToAllTables, and a
Buckets map. Nothing existing changed shape, and the new column is nullable and additive, so it migrates onto
an existing registry.

Verification

Real PostgreSQL and real SQL Server, not mocks.

  • 5 new PostgreSQL tests, all 5 red on 9.19.1 and green here — covering in-batch bucketing, sequential
    bucketing, non-destructive widening, co-tenant survival on drop, and release-on-last-member.
  • 5 new SQL Server tests. These exercise new API so they cannot compile against 9.19.1; the underlying defect
    was instead reproduced on 9.19.1 through the old API in the documented sequential shape, which allocated two
    distinct ordinals where one was expected.
  • Full suites green: Weasel.Postgresql 780/783 and Weasel.SqlServer 323/331 (skips pre-existing).

Downstream: JasperFx/wolverine#3683 adopts this and adds bucketing compliance coverage on both engines.

…tition (#391)

Both managed-partition strategies advertised multi-tenant "bucketing" -- several small
tenants sharing one physical partition to stay clear of per-table partition ceilings --
but neither could express it end to end. Reproduced against real PostgreSQL and SQL Server.

PostgreSQL (ManagedListPartitions)

AddPartitionToAllTables emitted one single-value ListPartition per value, so two values
sharing a suffix produced two CREATE TABLE IF NOT EXISTS statements against the same
partition table name. The first created it with only its own value and the second was
silently swallowed, so the partition never accepted the second value and its first write
died with 23514 "no partition of relation found for row".

The additive path now groups by SANITIZED suffix (that is what names the physical table)
and carries the bucket's full membership. Since CREATE TABLE IF NOT EXISTS cannot alter an
existing bound, widening an already-present partition is DETACH + re-ATTACH inside ONE
transaction -- a failure between them would orphan the partition and leave every tenant in
the bucket unroutable.

DropPartitionFromAllTablesForValue resolved a value to its suffix and then dropped BY
SUFFIX, deleting every registry row in the bucket and DROPping the shared partition table.
Removing one small tenant therefore silently destroyed its co-tenants' rows. It now narrows
the bucket -- deleting only the departing value's rows, then re-binding to the remaining
members -- and releases the partition only with the last member.

SQL Server (ManagedTenantPartitions)

Bucketing worked inside a single batch (the caller supplied the shared ordinal itself) but
silently did not across calls -- the natural tenant-onboarding shape. The registry persisted
only tenant_id -> ordinal, so a brand-new tenant had no way to discover which ordinal its
bucket already owned and each call allocated a fresh one; the tenants never actually shared.

The registry gains a nullable `bucket` column plus a bucket-aware AddPartitionsToAllTables
overload taking tenant -> bucket. Deliberately a COLUMN rather than a pseudo-tenant row: a
row would keep the ordinal referenced forever, defeating release-on-last-member and
TenantDropBehavior.DeleteData. A bucket is forgotten once its last member is dropped.

Coverage: 5 new PostgreSQL tests (all red on the prior code) and 5 new SQL Server tests.
Full suites green -- Weasel.Postgresql 780/783, Weasel.SqlServer 323/331 (skips pre-existing).
New API surface on ManagedTenantPartitions (bucket-aware AddPartitionsToAllTables + Buckets),
so a minor bump rather than a patch. Wolverine's GH-3683 adoption depends on this version.
jeremydmiller added a commit to JasperFx/wolverine that referenced this pull request Jul 28, 2026
…H-3683)

Tenant "bucketing" -- registering several small tenants against one partition suffix so
they share a physical partition -- is documented on the EF Core multi-tenancy page and
exposed via ConjoinedTenancyOptions.PartitionPerTenant(p => p.AllowPartitionSharing = true),
but it did not work on either engine. It had no test coverage, which is why this went
unnoticed. Both defects were reproduced against real PostgreSQL and SQL Server.

The root cause was in Weasel and is fixed by JasperFx/weasel#392 (9.20.0):

  - PostgreSQL emitted one single-value ListPartition per tenant, so the second member of
    a bucket was swallowed by CREATE TABLE IF NOT EXISTS and its first write failed with
    23514. Widening an existing partition needs DETACH + re-ATTACH, which IF NOT EXISTS
    cannot express.
  - SQL Server's ordinal registry had no column for the bucket, so a tenant joining an
    existing bucket in a later call could not discover its ordinal.

Wolverine changes here:

PostgresqlTenantPartitioning needs no change at all -- it hands Weasel the tenant -> suffix
map and the fixed Weasel does the right thing.

SqlServerTenantPartitioning was resolving the bucket's ordinal itself out of the tenant ->
ordinal map, which is blind to buckets: for a brand new tenant the lookup matched nothing
and it allocated a fresh ordinal, so sequentially registered members silently never shared.
It now delegates to Weasel's bucket-aware overload, and its AllowPartitionSharing guard
spans calls via the persisted bucket map, matching what PostgreSQL already did.

Compliance coverage on BOTH engines for the acceptance criteria on GH-3683: members
registered together and separately land in one physical partition, and dropping one member
leaves the others reading, writing, and holding their data. That last one covers GH-3686,
the co-tenant data loss found alongside this.

Docs: the bucketing sample showed the sequential pattern without enabling
AllowPartitionSharing, so as written it would have thrown. The config sample now enables it,
and a new Tenant Bucketing section covers when to reach for it, the drop semantics, and the
9.20.0 requirement.

Verified: EfCoreTests.MultiTenancy 193/193, SqlServerTests 388/390 (2 skipped),
full wolverine.slnx Release build clean at 0 warnings.

NOTE: requires Weasel 9.20.0 on nuget.org; verified locally against a packed build.
@jeremydmiller
jeremydmiller merged commit b7d5edc into master Jul 28, 2026
23 checks passed
@jeremydmiller
jeremydmiller deleted the weasel-391/managed-partition-bucketing branch July 28, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Managed partition bucketing is not expressible: list partitions take one value, tenant ordinals have no bucket key

1 participant