fix: repopulate natural-key lookup table on projection rebuild (#259)#261
Merged
Conversation
Parity with marten#4788/#4793. NaturalKeyProjection maintains pc_natural_key_X via the inline-append path (driven by newly-appended StreamActions). The async daemon rebuild replays persisted events without appending streams, so the table was never repopulated on rebuild and FetchForWriting/FetchLatest by natural key missed after a rebuild. Two hook points (mirroring the Marten fix): - TeardownProjectionStateAsync: when the source carries a NaturalKeyDefinition, also wipe pc_natural_key_X so the rebuild starts clean. - StartProjectionBatchAsync (Rebuild mode): look up the source by shard name and, when it has a NaturalKeyDefinition, feed range.Events per tenant through the new NaturalKeyProjection.QueueUpsertsForEvents — routed via batch.SessionForTenant so the upserts flush in the same batch transaction as the rebuilt snapshots. NaturalKeyProjection refactored to share one per-event operation builder between ApplyAsync (StreamAction-driven) and QueueUpsertsForEvents (IEvent-driven, pulling stream id/key + tenant off the event). The shared builder keeps the Archived-event branch; archived streams are correctly excluded from rebuild (the loader filters is_archived = 0), so their natural-key rows are not re-emitted. Tests: table repopulated on single- and multi-stream rebuild; FetchForWriting by natural key works after rebuild; teardown wipes the table and archived streams are excluded from the rebuilt lookup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #259. Parity with marten#4788 / marten#4793.
Bug
NaturalKeyProjectionmaintains thepc_natural_key_Xlookup table via the inline-append path, which drives off newly-appendedStreamActions. The async-daemon rebuild replays already-persisted events without appending streams, soApplyAsyncnever fired during a rebuild — the table stayed empty afterward andFetchForWriting/FetchLatestby natural key missed.Fix (two hook points, mirroring the Marten fix)
TeardownProjectionStateAsync— when the projection source carries aNaturalKeyDefinition, also wipepc_natural_key_X(rides the sameDELETE FROMloop as the doc tables) so the rebuild starts from a clean slate.StartProjectionBatchAsyncinShardExecutionMode.Rebuild— look up the source byrange.ShardName.Name; when it has aNaturalKeyDefinition, feedrange.Eventsper tenant through the newNaturalKeyProjection.QueueUpsertsForEvents. Operations are routed throughbatch.SessionForTenant(tenantId)so they flush in the same batch transaction as the rebuilt snapshots.NaturalKeyProjectionwas refactored soApplyAsync(StreamAction-driven) andQueueUpsertsForEvents(raw-IEvent-driven, pulling stream id/key + tenant off the event) share one per-event operation builder.Archived-stream delta
The shared builder keeps the
Archived-event branch. Polecat's rebuild event loader filtersis_archived = 0, so archived streams' events are not replayed and their natural-key rows are not re-emitted on rebuild — correct, since an archived stream shouldn't resolve as an active natural-key lookup. Covered by a dedicated test.Tests (
Bug_259_natural_key_rebuild)DELETE FROM pc_natural_key_*→ rebuild → table has 1 row andFetchForWritingby natural key succeeds (the exact repro from the issue);Regression sweep (natural-key + Daemon + Projections): 218 passed / 0 failed.
🤖 Generated with Claude Code