release prep: 9.15.2#4951
Merged
Merged
Conversation
…tion sweep Both fixes come out of the same 512-tenant-database production deployment. GH-4946: the batch BulkInsertEventsAsync overloads ran a full schema apply on every call -- and ApplyAllConfiguredChangesToDatabaseAsync applies across every database in the store, so a sharded store paid one apply per database per batch. Import throughput collapsed to ~17 events/s (the streaming overload does >3,000) with the connection pool parked on Weasel's partition-introspection query. The apply is now skipped entirely under AutoCreate.None, where it is a no-op by contract, and otherwise runs at most once per database the import touches. GH-4944: AddPartitionToAllTables walked the calling store's StoreOptions, so a provisioning tool that did not register every document type silently under-provisioned -- and the tenant then failed with 23514 on first write to the missing partition. The sweep is now database-driven, enumerating tenant list-partitioned tables from the catalog and scoped to the store's own schemas. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Patch release. Both fixes come out of the same 512-tenant-database production deployment reported by @erdtsieck.
#4946 — bulk event insert ran a full schema apply per batch (#4949)
The batch
BulkInsertEventsAsyncoverloads opened withStorage.ApplyAllConfiguredChangesToDatabaseAsync()on every call. That is not a cheap check — it callsTenancy.BuildDatabases()and runs a full schema delta across every database in the store. So a sharded store paid one apply per database, per batch: on the reporter's 512-database store, a ~1,000-event batch triggered 512 schema applies.Import throughput collapsed to ~17 events/s (the streaming overload, which has no such call, does >3,000/s) and the connection pool filled with backends parked on Weasel's partition-introspection query. A 686k-event tenant projected to ~11 hours.
The apply is now:
AutoCreateisNone— it is a no-op by contract there, so only the introspection cost remained; andIMartenDatabase.Identifier.The shared apply deliberately does not take a caller's
CancellationToken: the first caller to arrive owns the in-flight task every concurrent caller for that database awaits, so binding it to one caller's token would let a single cancelled batch fail sibling batches that were never cancelled. Each caller applies its own token at the await site instead.The document bulk-insert path is unaffected — it routes through the ordinary per-feature
EnsureStorageExistsAsyncthat Weasel memoizes, not a full-store delta.#4944 — database-driven tenant partition sweep (#4950)
AddPartitionToAllTableswalked the calling store'sStoreOptionsto decide which tables needed a list partition for a new tenant. A provisioning tool built on a store that does not register every document type therefore silently under-provisioned, and the tenant later failed with a Postgres23514check-constraint violation on first write to the missing partition. The workaround — "the provisioning tool must register every document type" — re-created schema knowledge in a second place and drifted as types were added.The sweep now enumerates tenant list-partitioned tables from the database catalog, so a partially-registered store still provisions every partitioned table.
Scoping is enforced inside the catalog query, not after the fact:
AllSchemaNames()only, so foreign tables in a shared database are never touched;tenant_id. This is the filter that matters most: it keeps the sweep off Marten's own non-tenant list partitioning (UseArchivedStreamPartitioningkeysmt_eventsonis_archived, andByList()keys on its own field);ByExternallyManagedListPartitions()are subtracted.Opt-out via
SweepPartitionedTablesFromDatabase(default on). No Weasel change required.Known limitation, documented: a document type registered into a schema the calling store has never heard of stays invisible to the schema filter — a store cannot own a schema it does not know exists. Single-schema stores (the default, and the reporter's shape) are fully covered.
🤖 Generated with Claude Code