Opt-in: gate new-projection-version side-effects behind the prior version's high-water (blue/green) — bounded rebuild to N, then Continuous

[jeremydmiller]

Summary

Add an opt-in behavior for blue/green projection-version deploys: when a new version of an existing projection is deployed, it should not fire side-effects (RaiseSideEffects → raised events / published messages) while it replays history the previous version already processed. It should only begin emitting side-effects once it advances past where the previous version left off.

Mechanism: detect the fresh-deploy case, resolve the prior version's progression (N), run a bounded Rebuild to N (Rebuild already suppresses side-effects), then hand off to Continuous from N (side-effects resume for events > N). Strictly opt-in — it's a significant behavior change from today.

Downstream of the CritterWatch blue/green discussion in JasperFx/CritterWatch#77 (and its sibling #74).

Motivation

Real user pain (CritterWatch#77): during blue/green deploys, projection versions are bumped. The new version starts running asynchronously and executes side-effects while catching up over already-processed history — publishing messages the old version already published. A pure rebuild avoids this (rebuild suppresses side-effects) but ends "caught up"; a freshly-deployed new version running live does not. Users are hand-rolling a static HasBeenMarkedReadyOnceSinceDeployment flag inside the projection to gate side-effects. That's the smell that says this belongs in the framework.

Trigger (all four must hold)

1. The projection opts in (new flag, below).
2. ProjectionBase.Version > 1.
3. A prior-version progression row exists with progress (e.g. Trips:V2:All at seq 1,000).
4. The new version has no progression of its own yet (fresh deploy, not a resume).

If any fail (no predecessor, or this version already has progress), behavior is unchanged.

Why this mechanism (clean boundary, reuses existing semantics)

A bounded rebuild makes N a range boundary, not a mid-slice cut: Phase 1 rebuilds [0…N] (side-effects suppressed, aggregate state correct), Phase 2 runs Continuous from N+1 (side-effects fire). No event slice straddles N, so no change to the RaiseSideEffects(operations, slice) contract is needed, and it maps onto the existing "rebuild = no side-effects" mental model. (An alternative per-event sequence predicate inside the side-effect gate at AggregationRunner.cs:270/310 was considered but rejected — a slice batches all of one aggregate's events across the range and can straddle N, forcing a contract change. The bounded-rebuild boundary avoids that entirely.)

Implementation shape (mostly composition of existing parts)

All in shared JasperFx.Events — Marten and Polecat both inherit it (Polecat's PolecatProjectionDaemon is a thin subclass of JasperFxAsyncDaemon, and message side-effects route through the shared AggregationRunner).

1. Opt-in flag on ProjectionBase (carried to every projection type), alongside Version. Working name GateSideEffectsBehindPriorVersion (bool, default false) — final name maintainer's call.
2. Detect + resolve N — at startup, when the trigger holds, resolve the highest prior version's progression. The data already survives the version bump: the version is baked into the progression-row PK (ShardName.Identity = Trips:V2:All vs Trips:V3:All are distinct rows; ShardName.cs:54-63, Marten PK EventProgressionTable.cs:19), so V2's mark is readable via ProjectionProgressFor/AllProjectionProgress. Nothing reads across versions today — that cross-version lookup is the one genuinely new read.
3. Bounded rebuild to N — already supported: pass N as the mark/highWaterMark into rebuildAgent/ReplayAsync instead of Tracker.HighWaterMark (JasperFxAsyncDaemon.cs:814). The terminal condition (SubscriptionAgent.cs:354) already compares LastCommitted against the seeded HighWaterMark field, so it stops at N with no terminal-logic change. Precedent: the per-tenant rebuild path already passes a custom ceiling this way (JasperFxAsyncDaemon.cs:871).
4. Hand off to Continuous from N — for the opt-in path, skip the unconditional post-rebuild StopAndDrainAsync (JasperFxAsyncDaemon.cs:833) and start the same shard Continuous via tryStartAgentAsync(agent, Continuous) (:112). DetermineStartingPositionAsync reads the persisted progress (now N) and resumes from N for the default CatchUp strategy (AsyncOptions.cs:264-265). Precedent: the optimized-rebuild path already runs this continuous-from-ceiling hand-off (SubscriptionAgent.cs:341-348).

Net new code: an opt-in bounded-rebuild-then-continue orchestration + the cross-version N resolver + the flag. The positioning, ceiling, and hand-off machinery already exist.

Caveats / open implementation points

• Optimized replay executor. The optimized IReplayExecutor path (gated at SubscriptionAgent.cs:215) is implemented in the store packages (Marten/Polecat), not JasperFx.Events. Those executors must honor the custom ceiling N too (or the opt-in path must use the non-optimized replay). Needs a check on each store's executor.
• Position strategy must not be FromPresent — FromPresent ignores the persisted row and jumps to the live high-water (AsyncOptions.cs:240-248), which would skip N. Opt-in should be incompatible with / override FromPresent, or warn.
• Overlap with a still-running old version (accepted compromise). N is snapshotted at the new version's start. In blue/green the old version keeps advancing until drained, so the new version's Continuous phase will re-fire side-effects in the window (N, old_final] for events the old version also emitted. This is accepted for v1 ("up to the original point"); closing the overlap is the job of the deploy-drain coordination tracked in JasperFx/CritterWatch#74.

Polecat parity

Free for the message-side-effect case (the relevant one): Polecat consumes the shared daemon + AggregationRunner and supports message side-effects via PolecatProjectionBatch.PublishMessageAsync. Polecat-side work would only be needed for event-emitting side-effects (Polecat stubs QuickAppendEvents as no-ops) or a new persisted progression column — neither is required by this design (it derives N from existing rows). The optimized-executor ceiling caveat above applies to Polecat's executor.

Downstream

• JasperFx/CritterWatch#77 — the originating blue/green side-effect-suppression ask.
• JasperFx/CritterWatch#74 — sibling (defer consumption / drain old version); closes the overlap window above.
• CritterWatch will add observability for the latch ("version V3 warming — side-effects suppressed until seq N, X behind").

[jeremydmiller]

Implementation is up in #520. Downstream adoption + testing-plan issues filed: JasperFx/marten#4963 and JasperFx/polecat#326.