Skip to content

High-water allocation fence: read is_called so mark 0 is fenceable (#5091) - #5092

Merged
jeremydmiller merged 2 commits into
masterfrom
fix/5091-is-called-allocation-fence
Jul 30, 2026
Merged

High-water allocation fence: read is_called so mark 0 is fenceable (#5091)#5092
jeremydmiller merged 2 commits into
masterfrom
fix/5091-is-called-allocation-fence

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #5091. Third and last piece of the #5090 chain; the other two shipped upstream and are adopted here.

The fence could never fire at mark 0

#5057 added the allocation fence so a provably-quiescent idle in transaction session can be ruled out as a candidate reserver of a stale sequence gap instead of pinning the high water mark forever. findAllocationFence(mark) looks for a recorded poll where the sequence was at or below the stuck mark — but it compared against the reserved ceiling (mt_events_sequence.last_value), and Postgres reports last_value = 1 for a sequence nothing has drawn from yet. No poll can ever report a value at or below a mark of 0, so a leading gap — the mark pinned at 0 under a hole at the very start of the sequence — was never fenceable, and any permanently idle open transaction held it forever. That is the reporter's shape in #5090: Daemon high water detection is holding before the sequence gap above 0, with the #5057 exclusion inert on 9.21.0.

is_called distinguishes "1 has been handed out" from "1 is the next value to hand out", which yields the highest allocated sequence number — 0 for a pristine sequence. The fence now compares against that.

This does not weaken the proof: everything above the allocated high was handed out strictly after the poll that observed it, which is the same monotonicity argument #5057 relies on. Unchanged in both directions:

  • StuckGapObservation.ReservedCeiling still bounds how far a proven-dead skip may advance, and still comes from last_value.
  • Per-tenant partitioned stores have no store-global allocation reading (tenants draw from their own sequences), so AllocatedSequenceHigh is null there and they keep today's conservative no-fence behaviour.
  • A detector that starts up over a gap that already exists still has no fence, and still holds.

Tests

DaemonTests/Bugs/Bug_5091_allocation_fence_at_mark_zero.cs:

  • a_dead_leading_gap_skips_despite_an_idle_advisory_lock_session — the fix.
  • a_live_reserver_of_the_leading_gap_still_holds_the_mark — fencing mark 0 must not become a licence to skip live appends.
  • a_detector_that_never_saw_the_pristine_sequence_still_holds_conservatively — the documented degradation is intact.

The first two are verified RED before the fix (mark stuck at 0 in both); the third passes either way, as intended.

Dependencies — this is what actually fixes #5090

Weasel 9.20.2 also carries weasel#399/#400, a regression found while making this bump and worth flagging: 9.18.0's computed-column delta detection routed every column through TableColumn.MatchesForDelta, where a bare Equals(actual) binds to the protected non-virtual Equals(TableColumn) overload and bypasses subclass overrides of Equals(object). That override is exactly how RevisionColumn tolerates an existing bigint mt_version (#4614/#4742), so Bug_4614's assert-check test fails with an empty change set on every Weasel from 9.18.0 through 9.20.1. 9.20.2 is the first version this repo can move to.

Docs

Notes that a process starting more than one daemon-hosting IHost over its lifetime — the usual xUnit integration-suite shape — should prefer Events.UseAdvisoryLockTransaction = false, since a session-scoped leadership lock holds no open transaction for gap detection to treat as a possible in-flight append. markdownlint and cspell clean.

Verification

On the published packages, net10.0: CoreTests 494/494, DaemonTests 260/260, EventSourcingTests 1478/1478 (7 skipped).

🤖 Generated with Claude Code

jeremydmiller and others added 2 commits July 30, 2026 09:10
…sequence

The #4953/#5057 allocation fence was built from the sequence's RESERVED ceiling
(mt_events_sequence.last_value), and Postgres reports last_value = 1 for a
sequence nothing has drawn from yet. No poll could ever report a value at or
below a stuck mark of 0, so findAllocationFence(0) always returned null: a
leading gap — the mark pinned at 0 under a hole at the very start of the
sequence — could never be fenced, and any permanently idle open transaction held
it forever. That is the shape reported in #5090 ("holding before the sequence
gap above 0"), where the quiescent-session exclusion was inert on 9.21.0.

The statistics reading now also carries is_called, which distinguishes "1 has
been handed out" from "1 is the next value to hand out", giving the highest
ALLOCATED sequence number — 0 for a pristine sequence. The fence compares
against that instead, which makes mark 0 fenceable without weakening the proof:
everything above the allocated high was handed out strictly after the poll that
observed it. The reserved ceiling still bounds how far a proven-dead skip may
advance (StuckGapObservation.ReservedCeiling is unchanged), and per-tenant
partitioned stores report no allocation reading at all, so they keep today's
conservative no-fence behaviour.

Also documents that a process starting more than one daemon-hosting IHost over
its lifetime should prefer Events.UseAdvisoryLockTransaction = false, since a
session-scoped leadership lock holds no open transaction that gap detection has
to treat as a possible in-flight append.

Tests cover the dead leading gap skipping past an idle advisory-lock session,
a live reserver of the leading gap still holding the mark, and a detector that
never saw the pristine sequence still holding conservatively. The first two are
verified RED before the fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
JasperFx 2.36.3 (#592/#593): ProjectionCoordinatorBase.StartAsync cancels
AND drains any existing leadership loop instead of disposing the old
CancellationTokenSource without cancelling it, so a bare ResumeAsync can no
longer orphan a live loop that outlives StopAsync.

Weasel 9.20.2 (weasel#396/#397): AdvisoryLock can no longer strand a handle
acquired after DisposeAsync drained — the actual cause of the permanently
'idle in transaction' advisory-lock session in #5090.

Weasel 9.20.2 also carries weasel#399/#400, found while making this bump:
9.18.0's computed-column delta detection made TableColumn.MatchesForDelta
compare through the non-virtual Equals(TableColumn), bypassing subclass
overrides of Equals(object). That is the seam RevisionColumn uses to tolerate an
existing bigint mt_version (#4614/#4742), so every Weasel from 9.18.0 through
9.20.1 fails Bug_4614's assert-check test with an empty change set. 9.20.2 is
the first version this repo can move to.

Verified on the published packages: CoreTests 494, DaemonTests 260,
EventSourcingTests 1478 — all green on net10.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 88a1cd4 into master Jul 30, 2026
10 checks passed
@jeremydmiller
jeremydmiller deleted the fix/5091-is-called-allocation-fence branch July 30, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment