Revert Task.Yield() from AsyncWriteJournal and SnapshotStore#8188
Merged
Aaronontheweb merged 2 commits intoApr 26, 2026
Conversation
…napshotStore (akkadotnet#8163) This reverts the Task.Yield() additions from PR akkadotnet#8163 in AsyncWriteJournal.ExecuteBatch and SnapshotStore.ReceiveSnapshotStore, while preserving the health check test improvements from that same PR. PR akkadotnet#8163 added `await Task.Yield()` before calling `WriteMessagesAsync` and `SaveAsync` inside their respective circuit breaker lambdas. The intent was to move expensive byte serialization off the actor's message-processing thread, which showed ~45% throughput improvement in benchmarks. However, this silently broke the implicit contract that persistence plugins relied on: that the synchronous preamble of `WriteMessagesAsync`/`SaveAsync` executes in actor context. Moving execution to the thread pool caused: 1. Plugins that access `Self` inside `WriteMessagesAsync` (e.g. Akka.Persistence.Sql, Akka.Persistence.EventStore) throw `NotSupportedException` because there is no active ActorContext on a thread pool thread. 2. Plugins that use non-thread-safe collections like `Dictionary<string, Task>` for write tracking (e.g. Akka.Persistence.Sql, Akka.Persistence.EventStore) are now subject to concurrent access from both the actor thread and thread pool threads, causing `InvalidOperationException` or silent data corruption. 3. Plugins that send messages to subscribers after writes complete (e.g. Akka.Persistence.Redis) access shared actor state off the actor thread. The change was too blunt an instrument — it applied uniformly to all plugins via the base class, removing their ability to do any actor-thread setup before async work begins. Ironically, the plugins that benefit most from off-thread serialization (MongoDB, Azure Table Storage) don't access actor context at all, while the plugins that break (SQL, EventStore, Redis) already perform serialization off-thread in their async pipelines. A future version may reintroduce this optimization with a more surgical approach (e.g. opt-in property or Template Method pattern) that preserves the plugin threading contract.
5 tasks
3 tasks
Member
|
FWIW I should note that at least on my branch of akka.Persistence.Sql, after applying fixes to make 1.5.66 'work', 1.5.64 is still faster on the local benchmarks for heavy write load.... 1.5.64: 1.5.66: Leaving these as breadcrumbs before I lose them to confirm if the |
Member
|
1.5.57: |
Member
Author
|
Good data @to11mtm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Task.Yield()additions from PR Ensure WriteMessagesAsync/SaveAsync is called asynchronously in Async… #8163 inAsyncWriteJournal.ExecuteBatchandSnapshotStore.ReceiveSnapshotStoreAwaitAssertAsync,ExpectMsgAsync) from that same PR — those are independently correctWhy
PR #8163 added
await Task.Yield()before callingWriteMessagesAsync/SaveAsyncinside their circuit breaker lambdas to move serialization off the actor thread (~45% throughput gain in benchmarks). However, this silently broke the implicit contract that persistence plugins rely on — that the synchronous preamble of these methods executes in actor context.Affected plugins:
SelfinsideWriteMessagesAsync→NotSupportedExceptionoff actor thread; useDictionary<string, Task>for write tracking → concurrent access race conditionsTellto subscribers after writes complete → accesses shared actor state off-threadIrony: The plugins that benefit most from off-thread serialization (MongoDB, Azure Table Storage — they serialize on the actor thread before any async boundary) don't access actor context at all. The plugins that break (SQL, EventStore, Redis) already do serialization off-thread in their async pipelines and gain little from the
Task.Yield().A future version may reintroduce this optimization with a more targeted approach (opt-in property, Template Method pattern) that preserves the plugin threading contract.
Test plan
dotnet build src/core/Akka.Persistence/Akka.Persistence.csproj -c Release— compilesAkka.Persistence.Tests— 285 passed, 0 failed (net10.0)Akka.Persistence.TestKit.Tests— 37 passed, 0 failed (net10.0)