Repository navigation
Fault the projection batch when an operation cannot be configured (#5497) - #5498
Merged
jeremydmiller merged 1 commit intoSep 24, 2026
Conversation
…sperFx#5497) An exception thrown while an operation was appended to a page, such as a document serializer failing in WriteToParameter, was logged by the operation queue and dropped. The page kept the half-configured command, so the batch failed later on unrelated SQL (42883: text = integer) or committed without the document while the progression advanced. ProjectionUpdateBatch now keeps the first such exception, skips the operations after it, and rethrows it from WaitForCompletion before any page executes, so the daemon's error handling sees the real cause.
jeremydmiller
added a commit
that referenced
this pull request
Sep 24, 2026
…5504) #5498 made an operation that cannot be configured fault the batch instead of leaving a half-appended command on the page. The exception that now reaches the daemon is the real one, but it is the serializer's -- it names neither the document type nor the operation that was writing it, so the reader is still left inferring both from a stack frame. That was half of the "we lost time on this twice" in #5497. processOperationAsync now describes the failing operation (role, operation type, document type) onto the exception's Data -- the same channel MartenExceptionTransformer already uses to carry the NpgsqlCommand -- and logs it through the session logger next to the failure. The exception is deliberately NOT wrapped. GroupedProjectionExecution decides whether a shard failure is a real failure or a shutdown side effect by inspecting the exception, and the skip-poison-pill options key off exception type as well; a wrapper would change what both of those see for the sake of a better message. Describing and logging cannot throw over the top of the captured exception either, which is its own test. FailedOperationDataKey is static readonly rather than const on purpose: a const is inlined into the caller's IL, so an assembly compiled against one value would keep reading that key after Marten started writing another. Claude-Session: https://claude.ai/code/session_01481eiEZg1Bv6pu4DrgPRg3 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 24, 2026
This was referenced Oct 2, 2026
This was referenced Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5497.
When an operation throws while
ProjectionUpdateBatchappends it to a page (for example a document serializer failing inWriteToParameter), the operation queue'sBlocklogs the exception and moves on.OperationPage.Appendhas already started the command and bumpedCount, so the page keeps half an upsert. The batch then either fails on unrelated SQL (in our case42883: operator does not exist: text = integer, reported against the progression inserts), or commits without that document while the progression advances. The second outcome is silent data loss until a rebuild.Change:
processOperationAsynckeeps the first exception (ExceptionDispatchInfo) and skips the operations after it.WaitForCompletionrethrows it right after the queue drains, before patches are applied and before any page is built into a command, so the daemon's normal error handling sees the real cause.Tests:
Bug_5497_an_operation_that_cannot_be_configured_faults_the_batch:WaitForCompletion;Both are red on master and green with the change.
DaemonTests.Bugs,Aggregationsandrebuild_with_bulk_copyshow the same set of failures with and without the change on my machine (9 environmental ones, identical test names).