Three defects around extended progression tracking / mt_event_progression, found while building the clean-room repro for #537 (the daemon never persists heartbeat/agent_status — see that issue for the write-path gap itself). All verified on Marten 9.16.1 / JasperFx.Events 2.30.1 against Postgres 17.
1. SetEventStoreInstrumentation silently clobbers a direct EnableExtendedProgressionTracking = true
The documented Marten-side toggle does not work under AddMarten:
builder.Services.AddMarten(opts =>
{
opts.Connection(cs);
opts.Events.EnableExtendedProgressionTracking = true; // documented; silently lost
opts.Projections.Add(new CounterProjection(), ProjectionLifecycle.Async);
}).AddAsyncDaemon(DaemonMode.Solo);
Observed: store.Options.Events.EnableExtendedProgressionTracking == false at runtime, and the extended columns are never added to mt_event_progression.
Cause: AddMarten always registers SetEventStoreInstrumentation (MartenServiceCollectionExtensions.cs), whose IConfigureMarten.Configure runs at store build and unconditionally overwrites:
options.EventGraph.EnableExtendedProgressionTracking = ExtendedProgressionEnabled; // adapter default: false
options.EventGraph.AppendObserver = AppendObserver; // adapter default: null
So the only opt-in that works under DI is mutating the DI-registered IEventStoreInstrumentation singleton (which is what CritterWatch does). Same clobber applies to AppendObserver, and to SetEventStoreInstrumentation<T> for ancillary stores. The adapter should OR-combine / only apply when explicitly set.
2. Docs opt-in snippet throws InvalidCastException
docs/events/projections/async-daemon.md shows:
((IEventStoreInstrumentation)opts.Events).ExtendedProgressionEnabled = true;
EventGraph implements IEventStoreOptions / IReadOnlyEventStoreOptions but not IEventStoreInstrumentation, so this throws Unable to cast object of type 'Marten.Events.EventGraph' to type 'JasperFx.Events.IEventStoreInstrumentation' (verified on 9.16.1; master's EventGraph.cs still doesn't implement it). The docs also say EnableExtendedProgressionTracking "continues to work" — which item 1 shows is untrue under AddMarten.
3. last_updated is frozen at insert time for daemon shard rows
With an async projection advancing for 2 minutes:
Counters:All seq=48 last_updated=2:24:19 PM
Counters:All seq=391 last_updated=2:24:19 PM <- 2 minutes later, frozen
HighWaterMark seq=394 last_updated=2:26:19 PM <- advances fine
Cause: the daemon writes shard progress via InsertProjectionProgress (insert into ... (name, last_seq_id) values (?, ?) — last_updated filled by table default) and UpdateProjectionProgress (update ... set last_seq_id = ? where name = ? and last_seq_id = ?) — neither touches last_updated. Only HighWaterDetector goes through mt_mark_event_progression(), whose upsert does set last_updated = transaction_timestamp(). If freezing is by design it's a trap for anyone using last_updated as an advance signal (it looks exactly like a stalled projection); if not, UpdateProjectionProgress should set it.
Repro
Full standalone program in #537 (Marten 9.16.1, AddAsyncDaemon(DaemonMode.Solo), one async SingleStreamProjection, continuous appends, progression dumped every 15s).
Refs: #537, #519, CritterWatch#750.
Three defects around extended progression tracking /
mt_event_progression, found while building the clean-room repro for #537 (the daemon never persistsheartbeat/agent_status— see that issue for the write-path gap itself). All verified on Marten 9.16.1 / JasperFx.Events 2.30.1 against Postgres 17.1.
SetEventStoreInstrumentationsilently clobbers a directEnableExtendedProgressionTracking = trueThe documented Marten-side toggle does not work under
AddMarten:Observed:
store.Options.Events.EnableExtendedProgressionTracking == falseat runtime, and the extended columns are never added tomt_event_progression.Cause:
AddMartenalways registersSetEventStoreInstrumentation(MartenServiceCollectionExtensions.cs), whoseIConfigureMarten.Configureruns at store build and unconditionally overwrites:So the only opt-in that works under DI is mutating the DI-registered
IEventStoreInstrumentationsingleton (which is what CritterWatch does). Same clobber applies toAppendObserver, and toSetEventStoreInstrumentation<T>for ancillary stores. The adapter should OR-combine / only apply when explicitly set.2. Docs opt-in snippet throws
InvalidCastExceptiondocs/events/projections/async-daemon.mdshows:EventGraphimplementsIEventStoreOptions/IReadOnlyEventStoreOptionsbut notIEventStoreInstrumentation, so this throwsUnable to cast object of type 'Marten.Events.EventGraph' to type 'JasperFx.Events.IEventStoreInstrumentation'(verified on 9.16.1; master'sEventGraph.csstill doesn't implement it). The docs also sayEnableExtendedProgressionTracking"continues to work" — which item 1 shows is untrue underAddMarten.3.
last_updatedis frozen at insert time for daemon shard rowsWith an async projection advancing for 2 minutes:
Cause: the daemon writes shard progress via
InsertProjectionProgress(insert into ... (name, last_seq_id) values (?, ?)—last_updatedfilled by table default) andUpdateProjectionProgress(update ... set last_seq_id = ? where name = ? and last_seq_id = ?) — neither toucheslast_updated. OnlyHighWaterDetectorgoes throughmt_mark_event_progression(), whose upsert does setlast_updated = transaction_timestamp(). If freezing is by design it's a trap for anyone usinglast_updatedas an advance signal (it looks exactly like a stalled projection); if not,UpdateProjectionProgressshould set it.Repro
Full standalone program in #537 (Marten 9.16.1,
AddAsyncDaemon(DaemonMode.Solo), one asyncSingleStreamProjection, continuous appends, progression dumped every 15s).Refs: #537, #519, CritterWatch#750.