Skip to content

Add sharded multi-tenancy with database pooling#4211

Merged
jeremydmiller merged 5 commits intomasterfrom
feature/sharded-tenancy
Mar 27, 2026
Merged

Add sharded multi-tenancy with database pooling#4211
jeremydmiller merged 5 commits intomasterfrom
feature/sharded-tenancy

Conversation

@jeremydmiller
Copy link
Copy Markdown
Member

Summary

  • New ShardedTenancy ITenancy implementation for extreme scalability (targeting 200B+ events)
  • Distributes tenants across a pool of databases with conjoined tenancy + native PG LIST partitioning per tenant
  • Three built-in assignment strategies: hash (default), smallest database, explicit
  • Normalized database registry (mt_database_pool, mt_tenant_assignments) in master database
  • Admin API: AddTenantToShardAsync, AddDatabaseToPoolAsync, MarkDatabaseFullAsync
  • Advisory-locked concurrent-safe tenant assignment
  • Fluent configuration via MultiTenantedWithShardedDatabases()
  • Documentation in docs/configuration/multitenancy.md
  • Depends on Weasel 8.11 (Add sharded multi-tenancy abstractions weasel#231)

Test plan

  • 12 new sharded tenancy integration tests all passing
  • Full MultiTenancyTests suite: 128/128 passing, zero regressions
  • Existing managed partitioning tests: 8/8 passing
  • Existing bucketed sharding tests: 10/10 passing

🤖 Generated with Claude Code

jeremydmiller and others added 5 commits March 27, 2026 09:09
New ShardedTenancy ITenancy implementation that:
- Distributes tenants across a pool of databases (mt_database_pool)
- Tracks tenant-to-database assignments (mt_tenant_assignments)
- Creates native PG LIST partitions per tenant within each database
- Supports 3 assignment strategies: hash, smallest, explicit
- Uses advisory locks for concurrent-safe tenant assignment
- Provides admin API (AddTenantToShardAsync, AddDatabaseToPoolAsync, etc.)

Weasel abstractions (ITenantDatabasePool, ITenantAssignmentStrategy,
IDatabaseSizingStrategy) enable other CritterStack tools to manage tenants.

8/12 integration tests passing. Remaining 4 need schema migration ordering
fix for partition table creation in shard databases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fixed schema cleanup: ensure public schema exists before dropping mt_ objects
- Fixed MarkDatabaseFullAsync: seed databases before marking full so the pool
  row exists when the UPDATE runs
- Fixed findOrAssignTenantDatabaseAsync: seed databases before auto-assignment
  so the available database list is populated
- All 128 MultiTenancyTests pass including existing managed partitioning,
  bucketed sharding, and master table tenancy tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New section in multitenancy.md covering:
- Configuration with MultiTenantedWithShardedDatabases()
- Three assignment strategies: hash, smallest, explicit (with custom)
- Database registry tables (mt_database_pool, mt_tenant_assignments)
- Admin API for runtime tenant/database management
- Implicit vs explicit tenant assignment flow
- Async daemon integration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Revert ProjectReference to PackageReference for Weasel.Postgresql
- Update Weasel.Postgresql and Weasel.EntityFrameworkCore to 8.11.0
- Remove lib/weasel submodule and .gitmodules
- All 128 MultiTenancyTests pass (including 12 sharded tenancy tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- MD004: switch unordered lists from dashes to asterisks
- MD060: add spaces in table separator rows for proper pipe spacing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jeremydmiller
Copy link
Copy Markdown
Member Author

The tests all pass locally, we've got some kind of hopefully temporary Docker issue on GH actions

@jeremydmiller jeremydmiller merged commit 18199d5 into master Mar 27, 2026
1 of 6 checks passed
@jeremydmiller jeremydmiller deleted the feature/sharded-tenancy branch March 27, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant