Database-driven tenant partition sweep (#4944)

[jeremydmiller]

Fixes #4944.

Problem

AddPartitionToAllTables decided which tables needed a new tenant's list partition by walking the calling store's StoreOptions:

var tables = database.AllObjects().OfType<Table>()
    .Where(x => x.Partitioning is ListPartitioning list && list.PartitionManager == this)

So "all tables" really meant "all tables this StoreOptions happens to know about". Any process that provisions tenants from a partially-registered store — a conversion tool, an admin CLI, a seeding job, all of which typically register only the event types they need — silently under-provisioned. The registry row and the event partitions landed, the tenant looked provisioned, and the damage only surfaced when the application did its first document write into a partition that was never created: 23514 no partition of relation ....

Schema migration never heals this either, by design: document partitions are excluded from the table delta (IgnorePartitionsInMigration, #4706), so partition creation is exclusively the provisioning path's job — which is exactly why that path has to be complete regardless of who calls it.

The workaround so far was "the provisioning tool must register every document type", which re-creates the schema knowledge in a second place and drifts as document types are added.

Mechanism

Rather than trusting the caller's registrations, ask the database what is actually there. DatabaseScopedTenantPartitions now enumerates the tenant-list-partitioned parents straight out of the PostgreSQL catalog (pg_partitioned_table / pg_class / pg_inherits) and creates the tenant's partition on every one of them.

This is layered purely additively on top of the existing options-driven pass — CREATE TABLE IF NOT EXISTS <parent>_<suffix> PARTITION OF <parent> is idempotent, and tables the base pass already handled are skipped — so it is independent of which process happens to call it. Both provisioning entry points (AdvancedOperations.AddMartenManagedTenantsAsync and ShardedTenancy.createPartitionsForTenant) funnel through this one chokepoint, so both are fixed.

Combined with the #4942 idempotent repair, this also heals the "document type added after the tenant was provisioned" drift case.

Only the parent's Identifier is needed to emit the DDL, so the sweep never tries to reconstruct an unknown table's definition and never runs a table delta against it.

How schema scoping is enforced

This is the dangerous part, and it is deliberately narrow — a shard database is routinely shared with other applications, and a sweep that grabbed every partitioned table it could see would start bolting Marten tenant partitions onto a stranger's tables. Three filters, all enforced in the catalog query itself:

1. Schema — only schemas this store owns (database.AllSchemaNames(), i.e. the distinct schemas of the store's own registered objects, which always includes the DatabaseSchemaName where mt_tenant_partitions itself lives). Foreign schemas in a shared database are never touched.
2. Partition shape — only LIST-partitioned parents (partstrat = 'l') with exactly one partition key column (partnatts = 1) whose name is tenant_id. This is what keeps the sweep off Marten's own non-tenant partitioning: UseArchivedStreamPartitioning list-partitions mt_events/mt_streams on is_archived, and MultiTenantedWithPartitioning(x => x.ByList()) on a duplicated field list-partitions on that field — neither has a tenant_id key, so neither is swept. Hash- and range-partitioned tables are excluded outright, as are tables that are themselves partitions of another parent.
3. External management — any table the calling store does know about and has marked ByExternallyManagedListPartitions() is subtracted. "Marten, keep your hands off my partitions" is honored.

SweepPartitionedTablesFromDatabase (default true) opts back out to the old, options-only behavior.

Documented residual limitation

A document type registered into a non-default schema that the calling store does not register stays invisible to filter (1) — a store cannot own a schema it has never heard of. Single-schema stores (the overwhelming default, and the reporter's shape) are fully covered. This is called out in the XML docs on the sweep.

Scope note

Done entirely inside Marten — no Weasel change is required. Everything the sweep needs (ListPartition, IPartition.WriteCreateStatement, Table, IDatabase.AllSchemaNames()) is already public API, and DatabaseScopedTenantPartitions already shadows both AddPartitionToAllTables overloads. So this ships with 9.15.2 rather than gating on a Weasel release. Hoisting the enumeration primitive into ManagedListPartitions for other Weasel consumers remains available as a follow-up.

Tests

src/TenantPartitionedEventsTests/Sharded/Bug_4944_database_driven_partition_sweep.cs. The whole point is that provisioning happens from a store with a deliberately incomplete registration — a test that provisions from a fully-registered store proves nothing:

• tool_store_with_no_document_types_still_provisions_every_partitioned_table — the reporter's exact shape. The app store registers and applies three tenant-partitioned document types; a tool store registering zero of them provisions the tenant; all three partitions must exist, and the application must then be able to write all three doc types (pre-fix: 23514).
• partially_registered_store_still_provisions_the_types_it_does_not_know — a store registering a strict subset (1 of 3) must still provision all 3.
• never_touches_partitioned_tables_outside_the_stores_schemas_or_shape — the negative. Two decoys, both shaped to be swept if the filters were sloppy: foreign_app.their_ledger (the exact shape the sweep looks for — LIST on a single tenant_id column — but in a schema the store does not own) and public.other_app_ledger (in a schema the store does own, but list-partitioned on a non-tenant column). Neither may gain a single partition, while the store's own tables are still swept.
• opting_out_of_the_sweep_restores_the_old_under_provisioning_behavior — pins the pre-Sharded tenancy: database-driven partition sweep — enumerate partitioned tables from the database (pg_inherits) instead of the calling store's StoreOptions #4944 behavior behind the toggle, and doubles as proof the other tests genuinely exercise the new sweep: with the sweep off, the tool store under-provisions exactly as reported.

Results: TenantPartitionedEventsTests 236/236 (run twice), CoreTests partitioning 41/41, MultiTenancyTests 159/159.

Found by @erdtsieck at 512 tenant databases; see #4943.

🤖 Generated with Claude Code