Supported per-tenant projection lag read (#619) - #624
Closed
jeremydmiller wants to merge 1 commit into
Closed
Conversation
"How far behind is projection X, at its current version, for tenant T on
this database?" had no supported answer, so the correlation had been
reinvented three times -- the daemon's blue/green side-effect gate, Marten's
WaitForNonStaleDataAsync, and application code in the field -- each
rediscovering the same traps (marten#4761, marten#5161, marten#4797,
marten#5170).
- `ProjectionLag`: one value per (shard, tenant) cell, carrying the whole
`ShardName` (so a sliced projection doesn't collapse to one row), the
database identifier (so a 512-database fan-out stays attributable), and
`HasProgressionRow` as a real field rather than a `Sequence == 0` sentinel.
- `ProjectionLagCalculator`: the provider-neutral correlation, anchored on
registered sources at their current version. A missing row is fully behind,
not caught up; prior-version rows are never borrowed; non-shard bookkeeping
rows are excluded; each tenant is measured against its own high-water mark;
a store-global `:All` agent under a tenanted store keeps its own cell.
- `IEventDatabase.FetchProjectionLagAsync` (+ a ShardName-scoped overload) as
default interface methods over the single AllProjectionProgress round trip
that already exists -- no new SQL -- and `IEventStore<,>` overloads that
supply `AllShards()` as the registry.
The one thing the API cannot fix is documented on it: the tenant set is
discovered from the HighWaterMark:{tenant} rows, so a tenant that has never
had one written is invisible rather than "fully behind".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fuk1GybEEmohFmboJuM4Po
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #619. Stacked on #623 (the
HighWaterMark:{tenant}parsing fix) — base isgh/618-high-water-tenant-parsing; retarget tomainonce that merges. The dependency is real: without it, high-water rows routed throughShardNameattribute every tenant's mark to the store-global one.What ships
ProjectionLag— one value per (shard, tenant) cell, shaped per the issue:Carries the whole
ShardName(a projection sliced across custom shard keys reports one cell per slice instead of collapsing to an indistinguishable tuple);HasProgressionRowis a real field, not aSequence == 0sentinel;DatabaseIdentifierkeeps a 512-database fan-out attributable.ProjectionLagCalculator— the provider-neutral correlation, public so a caller that already holds both halves can run it without a second round trip. The rules, each pinned by a test naming the incident it comes from:HighWaterMark:{tenant}rows; each cell measured against its own tenant's mark (marten#4761).:Allagent under a tenanted store keeps its own cell against the store-global mark — without it that agent vanishes from the report (the marten#4761 follow-up).The read surface —
IEventDatabase.FetchProjectionLagAsync(...)plus aShardName-scoped overload, as default interface methods over the singleAllProjectionProgressround trip that already exists. No new SQL, and every existing store gets it without implementing anything.Deviation from the issue's proposed signature
The issue put both overloads on
IEventDatabasewith no registry argument. A database has no registry — "registered sources at their current version" lives on the store — so the database-level methods take the registeredShardNames, andIEventStore<TOperations,TQuerySession>carries overloads that supplyAllShards()for you:The scoped overload takes a
ShardName, not astring? tenantId, as the issue asked.ShardKey == Allmeans every slice, a null tenant means every tenant, andVersionis deliberately not matched — the registry only ever holds the current version, so a caller doesn't have to know what it is.Documented limitation
Stated in the XML docs, because the API can't fix it: the tenant set is discovered from the
HighWaterMark:{tenant}rows, so a tenant that exists but has never had one written produces no result at all — invisible rather than "fully behind". Cross-reference a store-supplied tenant list where you have one.Not done here
The acceptance criterion — rewriting Marten's
WaitForNonStaleDataAsyncisCaughtUpon top of this — is a Marten-side change and needs a JasperFx release first. Worth filing against Marten once this ships;resolvePriorVersionProgressAsyncin this repo is the second candidate consumer and is a follow-up.Tests
src/EventTests/Daemon/ProjectionLagTests.cs— 17 tests covering each rule above plus the two default-interface read paths. FullEventTestssuite passes (702).🤖 Generated with Claude Code
https://claude.ai/code/session_01Fuk1GybEEmohFmboJuM4Po