Skip to content

Fix managed LIST partitions destructively rebuilt under multi-database apply (marten#4706)#303

Merged
jeremydmiller merged 2 commits into
masterfrom
fix/4706-managed-partition-migration
Jun 9, 2026
Merged

Fix managed LIST partitions destructively rebuilt under multi-database apply (marten#4706)#303
jeremydmiller merged 2 commits into
masterfrom
fix/4706-managed-partition-migration

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Fixes the Weasel side of JasperFx/marten#4706. Bumps to 9.1.1.

Problem

A single ManagedListPartitions instance is shared across every database in a store. Under sharded multi-tenancy each database has its own mt_tenant_partitions contents, and Marten's ApplyAllConfiguredChangesToDatabaseAsync runs the databases in parallel. The manager's one-shot _hasInitialized cache therefore held the first database's partition set, so every other database's table diff saw the existing per-tenant partitions as "unexpected" and destructively rebuilt the table (create <t>_temp as select * → drop → recreate → copy), failing with 23514: no partition of relation found for row.

Fix

  • Per-database load cache. Replace the one-shot _hasInitialized bool with _initializedFor, keyed on the connection's DataSource/Database, so InitializeAsync reloads when pointed at a different database. Single-database stores still load exactly once. ForceReload() now clears the key.
  • Additive bypass. additivelyMigrateTablesForNewPartitions clears IgnorePartitionsInMigration locally so the explicit AddPartitionToAllTables path still creates partitions even when a consumer (Marten) sets that flag on managed-partition tables to exempt them from the generic schema diff.

This is the Weasel half; the Marten side (marten#4706) sets IgnorePartitionsInMigration on its Marten-managed partitioned tables so the generic diff returns None for them (no rebuild), while partition creation continues through the additive path enabled here.

Tests

managed_list_partitions gains:

  • a rebuild guard — a managed table with IgnorePartitionsInMigration set is not destructively rebuilt by the generic diff over its existing partitions (CreateMigrationAsyncNone);
  • an additive-bypass guardAddPartitionToAllTables still physically creates the child partition with the flag set.

Full partition + migration suite green (111).

🤖 Generated with Claude Code

jeremydmiller and others added 2 commits June 9, 2026 13:39
…e apply (marten#4706); bump 9.1.1

A single ManagedListPartitions instance is shared across every database in a store. Under sharded
multi-tenancy each database has its own mt_tenant_partitions contents, and ApplyAllConfiguredChanges
runs the databases in parallel — so the manager's one-shot `_hasInitialized` cache held the first
database's partition set, and every other database's table diff saw the existing partitions as
"unexpected" and destructively rebuilt the table (CREATE _temp / copy), failing with 23514.

- Key the load cache on the connection's database (`_initializedFor`) instead of a one-shot bool, so
  InitializeAsync reloads when pointed at a different database. Single-database stores still load once.
  ForceReload() now clears the key (was setting the one-shot true; no callers).
- additivelyMigrateTablesForNewPartitions clears IgnorePartitionsInMigration locally so the explicit
  AddPartitionToAllTables path still creates partitions even when consumers (Marten) set that flag on
  managed-partition tables to exempt them from the generic schema diff.

Tests: managed_list_partitions gains a rebuild-guard (no destructive rebuild when IgnorePartitionsInMigration
is set over existing partitions) and an additive-bypass guard (AddPartitionToAllTables still adds with the
flag set). Full partition/migration suite green (111).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…bump 9.1.2

9.1.1's per-database reload of ManagedListPartitions (_initializedFor key +
_partitions.Clear()) is regressive under sharded multi-tenancy. The previous
one-shot load meant _partitions accumulated the UNION of every tenant ever
added; sharded LAZY event-partition creation depends on that union, because the
manager is not registered as a per-shard initializer — when mt_streams/mt_events
is created lazily on a shard, WriteCreateStatement reads the union so the
tenant's partition is included. Clearing the set on reload broke that, building
the shard table with the wrong tenant set -> 23514 "no partition of relation
mt_streams found for row" (regressed 4 Marten sharded daemon/catch-up tests).

The per-DB reload was also unnecessary: the destructive rebuild on redeploy
(the actual #4706 symptom) is prevented by the consumer setting
IgnorePartitionsInMigration on its managed-partition tables, which makes
ListPartitioning.CreateDelta short-circuit to None regardless of the partition
set. So restore the one-shot _hasInitialized loader and keep ONLY the additive
bypass in additivelyMigrateTablesForNewPartitions (so explicit
AddPartitionToAllTables still creates partitions for tables carrying the flag).

Verified against Marten: new sharded redeploy repro RED->GREEN, full
TenantPartitionedEventsTests 182/182 and CoreTests partitioning 33/33 green;
Weasel managed_list_partitions 11/11 per TFM.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeremydmiller

Copy link
Copy Markdown
Member Author

Correction (9.1.1 → 9.1.2). The per-database reload that shipped in 9.1.1 (_initializedFor key + _partitions.Clear()) is regressive under sharded multi-tenancy: the prior one-shot loader accumulated the union of all tenants, and sharded lazy event-partition creation relies on that union (the manager isn't a per-shard initializer), so clearing it built shard tables with the wrong tenant set → 23514 on mt_streams. It was also unnecessary — the destructive rebuild on redeploy is prevented by the consumer setting IgnorePartitionsInMigration on managed-partition tables (CreateDeltaNone).

This branch now restores the one-shot loader and keeps only the additive bypass. Verified against Marten: sharded redeploy repro RED→GREEN, TenantPartitionedEventsTests 182/182, CoreTests partitioning 33/33, Weasel managed_list_partitions 11/11 per TFM. Bumped to 9.1.2.

@jeremydmiller jeremydmiller merged commit 9ba6850 into master Jun 9, 2026
22 checks passed
@jeremydmiller jeremydmiller deleted the fix/4706-managed-partition-migration branch June 9, 2026 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant