Marten-owned rolling range partitions for time-series tables (#5093)

[jeremydmiller]

Closes #5093. Depends on JasperFx/weasel#401, shipped in Weasel 9.21.0 (bumped here from 9.20.2).

Problem

PartitionOn(x => x.When, x => x.ByExternallyManagedRangePartitions()) was the only workable option for a time-series document table, and its contract is "Marten builds the partitioned parent, and the application writes the child-partition DDL itself."

Any table partitioned by time needs its partition set to move: provision next month, drop last year. RangePartitioning's declared-list delta reads both halves of that as drift — a moved window means actual has partitions the declaration no longer lists — which resolves to PartitionDelta.Rebuild. On a multi-gigabyte table that is unusable, so applications take the externally-managed door and hand-write CREATE TABLE … PARTITION OF / DROP TABLE.

Opting out of Weasel means opting out of its ordering and dependency management, not just its DDL generation. Field case in JasperFx/CritterWatch#886: a hand-rolled rebuild of a partitioned metrics table re-created it in a schema where mt_immutable_timestamptz had not been created yet, so an ordinary computed index over a DateTimeOffset member failed with 42883. The index was correct; bypassing Weasel was the defect — and that failure mode is available to any Marten user doing time-series partitioning today.

What this adds

opts.Schema.For<MetricsSample>()
    .Duplicate(x => x.BucketEnd)
    // Keep 12 months of history, provision 3 months ahead.
    .PartitionOn(x => x.BucketEnd,
        x => x.ByRollingRange(PartitionPeriod.Month, periodsAhead: 3, periodsBehind: 12));

PartitionPeriod covers Hour, Day, Week, Month, Year. Overloads also take a RollingWindowPolicy directly, or a pre-built ManagedRangePartitions — pass the same manager to several document types to roll all of their tables forward in one pass. A TimeProvider is injectable so the window is testable without waiting on the calendar.

How the two halves are driven

	Driven by	Why
Provision the leading edge	ordinary schema migration	with a manager attached CreateDelta is purely additive, so a rolled-forward window is a CREATE, never a rebuild
Retire the trailing edge	MartenActivator startup pass + Advanced.*	migration never removes data, so the retention drop has to be driven separately

The startup pass is gated on the same opt-in as the migration itself (ApplyAllDatabaseChangesOnStartup) — that is how a host says "Marten owns this schema", and dropping a partition is emphatically a schema change. It runs before AssertDatabaseMatchesConfigurationAsync, on purpose: once the clock crosses a period boundary the database is legitimately missing the new leading-edge partition, and asserting first would fail a deployment over a difference the pass is about to close.

For hosts whose processes outlive periodsAhead — an hourly window especially — AdvancedOperations gains ApplyRollingPartitionsAsync, plus RollPartitionsForwardAsync (additive only) and DropAgedRollingPartitionsAsync (retention only) for callers who want the halves separately.

Design notes

• Managers are discovered from the database's own schema objects rather than tracked on StoreOptions. That stays correct for any table that grows a rolling window later, and makes re-running the pass unconditionally idempotent.
• ByRollingRange asserts at configuration time that the partition key is a duplicated DateTime/DateTimeOffset member. A window keyed on anything else is not a window, and this turns what would be an opaque partition-bound error during the first migration into a message that names the member.
• A DEFAULT overflow partition is always created, so a row written outside the provisioned window is stored rather than rejected with a 23514.
• ByExternallyManagedRangePartitions() remains, now documented for what it is actually for: genuinely external ownership (pg_partman).

Acceptance criteria from the issue

• A rolling monthly document table needs no application-authored DDL
• Rolling the window forward never produces PartitionDelta.Rebuild — rolling_the_window_forward_is_additive_and_never_a_rebuild pins the diff at Update
• Aged partitions dropped as a policy outcome, retention reclaim O(1) — drops_partitions_that_have_aged_past_the_retention_floor
• Safe and idempotent when several nodes start concurrently — the_apply_pass_is_idempotent_and_safe_to_run_concurrently

Tests

10 new tests in src/CoreTests/Partitioning/rolling_range_partitioning.cs, driving the window with a FakeTimeProvider rather than the calendar: configuration shape, the declared window, initial creation of the whole window + DEFAULT, no-op re-apply, additive roll-forward with data survival, retention drop, the full host-startup pass through ApplyAllDatabaseChangesOnStartup, concurrent idempotency, overflow rows landing in DEFAULT, and the non-temporal partition key rejection.

Green locally on net9.0 (and the partitioning tests on net10.0): CoreTests 504, DocumentDbTests 1090, MultiTenancyTests 159, TenantPartitionedEventsTests 240, EventSourcingTests 1478 — the last four also serving as the regression gate on the Weasel bump.

Known consumer: JasperFx/CritterWatch hand-writes exactly this for mt_doc_metricssample and can delete that code.

🤖 Generated with Claude Code