Skip to content

release prep: 9.15.2#4951

Merged
jeremydmiller merged 1 commit into
masterfrom
release/9.15.2
Jul 14, 2026
Merged

release prep: 9.15.2#4951
jeremydmiller merged 1 commit into
masterfrom
release/9.15.2

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Patch release. Both fixes come out of the same 512-tenant-database production deployment reported by @erdtsieck.

#4946 — bulk event insert ran a full schema apply per batch (#4949)

The batch BulkInsertEventsAsync overloads opened with Storage.ApplyAllConfiguredChangesToDatabaseAsync() on every call. That is not a cheap check — it calls Tenancy.BuildDatabases() and runs a full schema delta across every database in the store. So a sharded store paid one apply per database, per batch: on the reporter's 512-database store, a ~1,000-event batch triggered 512 schema applies.

Import throughput collapsed to ~17 events/s (the streaming overload, which has no such call, does >3,000/s) and the connection pool filled with backends parked on Weasel's partition-introspection query. A 686k-event tenant projected to ~11 hours.

The apply is now:

  • skipped entirely when AutoCreate is None — it is a no-op by contract there, so only the introspection cost remained; and
  • otherwise run at most once per database the import actually touches, memoized per IMartenDatabase.Identifier.

The shared apply deliberately does not take a caller's CancellationToken: the first caller to arrive owns the in-flight task every concurrent caller for that database awaits, so binding it to one caller's token would let a single cancelled batch fail sibling batches that were never cancelled. Each caller applies its own token at the await site instead.

The document bulk-insert path is unaffected — it routes through the ordinary per-feature EnsureStorageExistsAsync that Weasel memoizes, not a full-store delta.

#4944 — database-driven tenant partition sweep (#4950)

AddPartitionToAllTables walked the calling store's StoreOptions to decide which tables needed a list partition for a new tenant. A provisioning tool built on a store that does not register every document type therefore silently under-provisioned, and the tenant later failed with a Postgres 23514 check-constraint violation on first write to the missing partition. The workaround — "the provisioning tool must register every document type" — re-created schema knowledge in a second place and drifted as types were added.

The sweep now enumerates tenant list-partitioned tables from the database catalog, so a partially-registered store still provisions every partitioned table.

Scoping is enforced inside the catalog query, not after the fact:

  • schema — the store's own AllSchemaNames() only, so foreign tables in a shared database are never touched;
  • partition shape — LIST strategy, single key column, named tenant_id. This is the filter that matters most: it keeps the sweep off Marten's own non-tenant list partitioning (UseArchivedStreamPartitioning keys mt_events on is_archived, and ByList() keys on its own field);
  • external management — tables marked ByExternallyManagedListPartitions() are subtracted.

Opt-out via SweepPartitionedTablesFromDatabase (default on). No Weasel change required.

Known limitation, documented: a document type registered into a schema the calling store has never heard of stays invisible to the schema filter — a store cannot own a schema it does not know exists. Single-schema stores (the default, and the reporter's shape) are fully covered.

🤖 Generated with Claude Code

…tion sweep

Both fixes come out of the same 512-tenant-database production deployment.

GH-4946: the batch BulkInsertEventsAsync overloads ran a full schema apply on
every call -- and ApplyAllConfiguredChangesToDatabaseAsync applies across every
database in the store, so a sharded store paid one apply per database per batch.
Import throughput collapsed to ~17 events/s (the streaming overload does >3,000)
with the connection pool parked on Weasel's partition-introspection query. The
apply is now skipped entirely under AutoCreate.None, where it is a no-op by
contract, and otherwise runs at most once per database the import touches.

GH-4944: AddPartitionToAllTables walked the calling store's StoreOptions, so a
provisioning tool that did not register every document type silently
under-provisioned -- and the tenant then failed with 23514 on first write to the
missing partition. The sweep is now database-driven, enumerating tenant
list-partitioned tables from the catalog and scoped to the store's own schemas.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 5c72ff7 into master Jul 14, 2026
9 checks passed
@jeremydmiller
jeremydmiller deleted the release/9.15.2 branch July 14, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant