Skip to content

refactor(journaling): separate durable state and journal APIs - #11319

Merged
ReubenBond merged 6 commits into
dotnet:mainfrom
ReubenBond:rb-refactor-unify-journaling-api-names
Sep 18, 2026
Merged

ReubenBond merged 6 commits into
dotnet:mainfrom
ReubenBond:rb-refactor-unify-journaling-api-names

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Sep 17, 2026 •

Copy link
Copy Markdown
Member

Problem

The Journaling API mixes application state management with the state-machine protocol and journal ownership. The existing GetOrCreateState helpers require the implementation interface, while the public durable collection interfaces expose application operations. Programmatic access and keyed dependency injection also follow different construction paths.

Solution

Introduce the grain-facing IDurableStateManager with typed GetOrAddState, lookup, and WriteStateAsync, plus helpers for dictionaries, lists, queues, sets, values, persistent state, and task-completion sources. Keyed injection and programmatic access converge on the same named state component. Register custom grain-state implementations through IServiceCollection.AddStateMachine<TState, TImplementation>, with constructor-injection and name-aware factory overloads.

Rename IJournaledState to IStateMachine and use WritePendingEntries/WriteSnapshot for its output methods. Keep IJournaledStateManager as an independent journal-owner contract: explicit state-machine registration and lookup, initialization, writes, deletion, disposal, and diagnostics. Remove unused DeepCopy and IDurableNothing, and align catalog, metadata, hosting, and persistent-state adapter names.

The default grain manager implements both contracts as one activation-scoped instance. It reuses the activation's services and enrolls during grain-bound construction. Ordinary Grain and IGrainBase implementations can inject the application interface directly, and synchronous activation-setup hooks can resolve managers and state components. The activation scope owns DI-created components and dependencies. Declare state components during construction or synchronous setup so recovery completes at SetupState before application use.

IJournaledStateManagerFactory.CreateStandalone(JournalId) returns a journal owner for explicitly supplied IStateMachine components. The caller constructs and registers those components, initializes the owner, writes state, and disposes the owner when finished. Component instances and their dependencies remain caller-owned; the journal owner releases its own processing resources. Standalone owners use the host's shared journal services, as DurableJobs does, while DI-backed GetOrAdd belongs to the grain-facing application interface.

Registration closes synchronously when initialization begins; recovery, queued failure barriers, and state retirement retain their existing guarantees. Update DurableJobs, provider consumers, documentation, snippets, samples, and the generated public API. Regenerate package/API-specific compatibility suppressions for the intentional alpha API changes.

Rationale

Durable* names describe application state, IStateMachine describes its implementation protocol, and Journal* names describe persistence infrastructure. Ordinary activation-scoped DI caching supplies grain manager identity; standalone components use explicit caller-owned construction. This keeps the two ownership models separate.

Built-in grain-state constructors retain registration to support .NET's native open-generic keyed DI registrations. The manager owns registry admission and resolves the canonical keyed instance. Custom constructors leave registration to the manager.

Persisted state names, stream identities, command tokens, format-key values, serialization IDs, and provider metadata keys remain stable.

Microsoft Reviewers: Open in CodeFlow

Copilot AI lite review requested due to automatic review settings September 17, 2026 23:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad API, lifecycle, and provider refactor requires final human review.

Review effort: Lite
Findings: None

What changed in this PR

Refactors Orleans Journaling to separate durable application state APIs from journal state-machine infrastructure while updating providers, Durable Jobs, tests, documentation, samples, and public APIs.

Changes:

  • Adds IDurableStateManager and typed durable-state helpers.
  • Renames state-machine, catalog, metadata, and hosting APIs.
  • Updates integrations, providers, tests, benchmarks, samples, and documentation.
File Summary
test/​Orleans.Journaling.Tests/​VolatileJournalStorageProviderTests.cs Updated journaling tests
test/​Orleans.Journaling.Tests/​UpstreamMainCompatibilityTests.cs Updated compatibility tests
test/​Orleans.Journaling.Tests/​StorageStreamingTests.cs Updated storage tests
test/​Orleans.Journaling.Tests/​StateManagerTests.cs Updated state manager tests
test/​Orleans.Journaling.Tests/​S3JournalStorageTests.cs Updated S3 tests
test/​Orleans.Journaling.Tests/​OrleansBinaryJournalBufferWriterTests.cs Updated binary buffer tests
test/​Orleans.Journaling.Tests/​OrleansBinaryCodecSnapshotTests.cs Updated snapshot tests
test/​Orleans.Journaling.Tests/​KeyedJournalingRegistrationTests.cs Updated keyed registration tests
test/​Orleans.Journaling.Tests/​JournalStorageTelemetryTests.cs Updated telemetry tests
test/​Orleans.Journaling.Tests/​JournalStorageCatalogTests.cs Updated catalog tests
test/​Orleans.Journaling.Tests/​JournalSnapshotInfrastructure.cs Updated snapshot infrastructure
test/​Orleans.Journaling.Tests/​JournalingTestBase.cs Updated test base
test/​Orleans.Journaling.Tests/​JournalCatalogRangeTests.cs Updated catalog range tests
test/​Orleans.Journaling.Tests/​JournalBufferWriterOwnershipTests.cs Updated buffer ownership tests
test/​Orleans.Journaling.Tests/​JournalBatchTests.cs Updated batch tests
test/​Orleans.Journaling.Tests/​IntegrationTestFixture.cs Updated integration fixture
test/​Orleans.Journaling.Tests/​FormatMigrationClusterTests.cs Updated migration tests
test/​Orleans.Journaling.Tests/​DurableStateAndTcsRecoveryTests.cs Updated recovery tests
test/​Orleans.Journaling.Tests/​DurableListDirectWriteTests.cs Updated durable list tests
test/​Orleans.Journaling.Tests/​DurableCollectionDirectWriteTests.cs Updated durable collection tests
test/​Orleans.Journaling.Tests/​AzureTableStorageJournalingProviderBuilderTests.cs Updated Azure builder tests
test/​Orleans.Journaling.Tests/​AzureTableJournalStorageTests.cs Updated Azure Table tests
test/​Orleans.Journaling.Tests/​AzureTableJournalStorageProviderTests.cs Updated Azure provider tests
test/​Orleans.Journaling.Tests/​AzureTableCodecRecoveryTests.cs Updated Azure codec tests
test/​Orleans.Journaling.Tests/​AzureBlobJournalStorageTests.cs Updated Azure Blob tests
test/​Orleans.Journaling.Tests/​AzureBlobCodecRecoveryTests.cs Updated Azure Blob codec tests
test/​Orleans.Journaling.Json.Tests/​JsonCommandCodecAdditionalTests.cs Updated JSON codec tests
test/​Orleans.Journaling.Json.Tests/​JsonCodecTests.cs Updated JSON tests
test/​Orleans.Journaling.Json.Tests/​JsonCodecSnapshotTests.cs Updated JSON snapshot tests
test/​Orleans.Journaling.Json.Tests/​CodecRecoveryTests.cs Updated JSON recovery tests
test/​Orleans.DurableJobs.Tests/​DurableJobs/​JournaledJobShardSignalTests.cs Updated Durable Jobs tests
test/​Orleans.DurableJobs.Tests/​DurableJobs/​JournaledJobShardManagerTests.cs Updated Durable Jobs tests
test/​Orleans.DurableJobs.Tests/​DurableJobs/​JournaledJobShardDiscoveryTests.cs Updated Durable Jobs tests
test/​Orleans.DurableJobs.Tests/​DurableJobs/​IJobShardManagerTestFixture.cs Updated Durable Jobs fixture
test/​Orleans.Core.Tests/​DurableJobs/​LocalDurableJobManagerTests.cs Updated core Durable Jobs tests
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageTests.cs Updated Redis tests
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageTelemetryTests.cs Updated Redis telemetry tests
test/​Extensions/​Orleans.Redis.Tests/​Journaling/​RedisJournalStorageCatalogTests.cs Updated Redis catalog tests
test/​Benchmarks/​Journaling/​JournalReplayContextFactory.cs Updated benchmark factory
test/​Benchmarks/​Journaling/​DurableListJournalBenchmarks.cs Updated list benchmarks
test/​Benchmarks/​Journaling/​DurableCommandReaderBenchmarks.cs Updated command benchmarks
test/​Benchmarks/​Journaling/​Azure/​AzureJournalScenario.cs Updated Azure benchmark scenario
test/​Benchmarks/​Journaling/​Azure/​AzureJournalRunnerTests.cs Updated Azure benchmark tests
src/​Redis/​Orleans.Journaling.Redis/​RedisJournalStorageProvider.cs Updated Redis provider
src/​Redis/​Orleans.Journaling.Redis/​RedisJournalStorage.cs Updated Redis storage
src/​Redis/​Orleans.Journaling.Redis/​README.md Updated Redis documentation
src/​Orleans.Journaling/​VolatileJournalStorage.cs Updated volatile storage
src/​Orleans.Journaling/​README.md Updated journaling documentation
src/​Orleans.Journaling/​JournalReplayContext.cs Updated replay context
src/​Orleans.Journaling/​JournalingHostingExtensions.cs Updated hosting extensions
src/​Orleans.Journaling/​JournalingGrainLifecycle.cs Updated grain lifecycle
src/​Orleans.Journaling/​JournaledStateManagerOptions.cs Updated manager options
src/​Orleans.Journaling/​JournaledStateManagerFactory.cs Updated manager factory
src/​Orleans.Journaling/​JournaledStateManagerBinding.cs Updated manager binding
src/​Orleans.Journaling/​JournaledPersistentState.cs Updated persistent state
src/​Orleans.Journaling/​JournalCatalogRange.cs Updated catalog range
src/​Orleans.Journaling/​JournalCatalogListOptions.cs Updated catalog options
src/​Orleans.Journaling/​IStateMachine.cs Introduced state-machine API
src/​Orleans.Journaling/​IJournalStorageConsumer.cs Updated storage consumer API
src/​Orleans.Journaling/​IJournalStorageCatalog.cs Updated storage catalog API
src/​Orleans.Journaling/​IJournalStorage.cs Updated storage API
src/​Orleans.Journaling/​IJournaledStateManagerFactory.cs Updated manager factory API
src/​Orleans.Journaling/​IJournaledStateManager.cs Updated advanced manager API
src/​Orleans.Journaling/​IDurableStateManager.cs Introduced durable state API
src/​Orleans.Journaling/​Formats/​OrleansBinary/​OrleansBinaryJournalFormat.cs Updated binary format
src/​Orleans.Journaling/​Formats/​Json/​JsonLinesJournalFormat.cs Updated JSON format
src/​Orleans.Journaling/​Formats/​Json/​JsonJournalHostingExtensions.cs Updated JSON hosting
src/​Orleans.Journaling/​DurableValue.cs Updated durable value
src/​Orleans.Journaling/​DurableTaskCompletionSource.cs Updated durable task source
src/​Orleans.Journaling/​DurableStateManagerExtensions.cs Added manager extensions
src/​Orleans.Journaling/​DurableSet.cs Updated durable set
src/​Orleans.Journaling/​DurableQueue.cs Updated durable queue
src/​Orleans.Journaling/​DurableNothing.cs Updated durable placeholder
src/​Orleans.Journaling/​DurableList.cs Updated durable list
src/​Orleans.Journaling/​DurableGrain.cs Updated durable grain
src/​Orleans.Journaling/​DurableDictionary.cs Updated durable dictionary
src/​Orleans.Journaling/​CompatibilitySuppressions.xml Updated compatibility suppressions
src/​Orleans.DurableJobs/​JournaledJobShardState.cs Updated Durable Jobs state
src/​Orleans.DurableJobs/​JournaledJobShardManager.cs Updated Durable Jobs manager
src/​Orleans.DurableJobs/​Hosting/​DurableJobsExtensions.cs Updated Durable Jobs hosting
src/​Orleans.DurableJobs/​DurableJobsStorageInspector.cs Updated storage inspector
src/​Azure/​Orleans.Journaling.AzureStorage/​README.md Updated Azure documentation
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureTableJournalStorageProvider.cs Updated Azure Table provider
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorageProvider.cs Updated Azure Blob provider
src/​Azure/​Orleans.Journaling.AzureStorage/​AzureBlobJournalStorage.cs Updated Azure Blob storage
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorageProvider.cs Updated S3 provider
src/​AWS/​Orleans.Journaling.S3/​S3JournalStorage.cs Updated S3 storage
src/​AWS/​Orleans.Journaling.S3/​README.md Updated S3 documentation
src/​api/​Orleans.Journaling/​Orleans.Journaling.cs Regenerated public API
samples/​JournalingAzureBlobJson/​README.md Updated sample documentation
samples/​JournalingAzureBlobJson/​JournalingAzureBlobJson/​Program.cs Migrated sample APIs
docs/​site/​src/​content/​docs/​grains/​journaling/​snippets/​journaling/​JournalingBasics.cs Updated documentation snippet
docs/​site/​src/​content/​docs/​grains/​journaling/​samples.md Updated samples documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​runtime-behavior.md Updated runtime documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​redis-journal-storage.md Updated Redis documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​index.md Updated journaling overview
docs/​site/​src/​content/​docs/​grains/​journaling/​durable-state.md Updated durable state documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​durable-jobs-migration.md Updated migration documentation
docs/​site/​src/​content/​docs/​grains/​journaling/​configuration.md Updated configuration documentation
docs/​site/​src/​content/​docs/​grains/​event-sourcing/​index.md Updated event-sourcing documentation

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage

Metric Pull request
Lines 82.78% (114,859 / 138,756)
Branches 72.04% (33,310 / 46,238)

Report-only conclusion: current-main baseline stale.

The newest successful coverage run tested 39ea499, not current main 598c0d2.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

Copilot AI review requested due to automatic review settings September 18, 2026 18:58
@ReubenBond
ReubenBond force-pushed the rb-refactor-unify-journaling-api-names branch from 4ddaa22 to 3946bc3 Compare September 18, 2026 18:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad journaling API, lifecycle, provider, and compatibility refactor requires final human review.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 18, 2026 19:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical provider compilation errors and a custom-state disposal leak remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity

Open (2)

Comment thread src/AWS/Orleans.Journaling.S3/S3JournalStorage.cs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad API, lifecycle, provider, and persistence-related refactor requires final human review.

Review effort: Lite
Findings: None

Resolved since last review (2)

Copilot AI review requested due to automatic review settings September 18, 2026 20:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad API, provider, lifecycle, and consumer refactor warrants final human review.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 18, 2026 22:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad API, lifecycle, provider, and consumer refactor warrants final human review.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 18, 2026 22:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad public API and lifecycle refactor requires final human review.

Review effort: Lite
Findings: None

Copilot AI review requested due to automatic review settings September 18, 2026 23:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The broad API, provider, integration, and documentation refactor warrants final human review.

Review effort: Lite
Findings: None

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The packaged Durable Jobs README still references the removed ListOptions API.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Low severity

Open (1)

Comment thread src/Orleans.DurableJobs/JournaledJobShardManager.cs
Copilot AI review requested due to automatic review settings September 18, 2026 23:53
@ReubenBond
ReubenBond merged commit 598c0d2 into dotnet:main Sep 18, 2026
67 checks passed
@ReubenBond
ReubenBond deleted the rb-refactor-unify-journaling-api-names branch September 18, 2026 23:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The package README documents a metadata-less recovery fallback that contradicts the implementation and recovery tests.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Low severity Correct metadata-less recovery format fallback documentation

src/​Orleans.Journaling/​README.md:66

This fallback description contradicts the implementation: JournaledStateManager.ProcessRecoveryBuffer uses the configured write format when metadata?.FormatKey is absent (JournaledStateManager.cs:802-804), and Recovery_MetadataLessJournal_UsesConfiguredFormat verifies that behavior. Saying metadata-less data is always treated as Orleans Binary can lead operators to configure a migration incorrectly; document the configured-format fallback instead. The same claim is repeated later in this README and should be corrected there too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants