Repository navigation
feat(durable-jobs): support journal provider migration - #11277
Merged
ReubenBond merged 3 commits intoSep 17, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
A critical storage-inspector accounting issue and a moderate Azure registration guidance issue remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (2)
What changed in this PR
Adds named journal-provider migration support for Durable Jobs, allowing new shards to use a write provider while existing shards drain from their original provider.
Changes:
- Adds provider-aware shard discovery, routing, execution, and cleanup.
- Adds full namespace inspection and migration guidance.
- Extends provider registrations, APIs, tests, samples, and playground support.
| File | Summary |
|---|---|
test/Orleans.Journaling.Tests/KeyedJournalingRegistrationTests.cs |
Named registration isolation tests |
test/Orleans.Journaling.Tests/AzureTableStorageHostingExtensionsTests.cs |
Azure Table hosting tests |
test/Orleans.Journaling.Tests/AzureTableStorageGrainJournalingProviderBuilderTests.cs |
Named Azure Table provider tests |
test/Orleans.DurableJobs.Tests/DurableJobs/JournaledJobShardProviderMigrationTests.cs |
Migration and failure-isolation coverage |
test/Orleans.DurableJobs.Tests/DurableJobs/JournaledJobShardManagerTests.cs |
Shard manager regression tests |
test/Orleans.Core.Tests/DurableJobs/LocalDurableJobManagerTests.cs |
Provider-aware job operations |
test/Orleans.Core.Tests/DurableJobs/DurableJobsOptionsTests.cs |
Provider option validation |
test/Extensions/Orleans.Redis.Tests/Journaling/RedisGrainJournalingProviderBuilderTests.cs |
Named Redis provider tests |
src/Redis/Orleans.Journaling.Redis/RedisJournalStorageOptionsValidator.cs |
Validates named Redis options |
src/Redis/Orleans.Journaling.Redis/RedisJournalStorageHostingExtensions.cs |
Registers Redis providers |
src/Redis/Orleans.Journaling.Redis/RedisGrainJournalingProviderBuilder.cs |
Builds named Redis providers |
src/Orleans.Journaling/HostingExtensions.cs |
Common named registration infrastructure |
src/Orleans.DurableJobs/README.md |
Durable Jobs migration guidance |
src/Orleans.DurableJobs/JournaledJobShardManager.cs |
Multi-provider discovery and routing |
src/Orleans.DurableJobs/JournaledJobShard.cs |
Preserves provider affinity |
src/Orleans.DurableJobs/IDurableJobsStorageInspector.cs |
Storage inspection API |
src/Orleans.DurableJobs/Hosting/DurableJobsOptions.cs |
Write and draining provider options |
src/Orleans.DurableJobs/Hosting/DurableJobsExtensions.cs |
Durable Jobs service registration |
src/Orleans.DurableJobs/DurableJobsStorageInspector.cs |
Full namespace inventory |
src/Orleans.DurableJobs/DurableJobsJournalProviders.cs |
Provider resolution |
src/Azure/Orleans.Journaling.AzureStorage/AzureTableStorageHostingExtensions.cs |
Named Azure Table registration |
src/Azure/Orleans.Journaling.AzureStorage/AzureTableStorageGrainJournalingProviderBuilder.cs |
Azure Table provider configuration |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobStorageHostingExtensions.cs |
Named Azure Blob registration |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobStorageGrainJournalingProviderBuilder.cs |
Azure Blob provider configuration |
src/Azure/Orleans.DurableJobs.AzureStorage/README.md |
Azure migration guidance |
src/Azure/Orleans.DurableJobs.AzureStorage/Hosting/AzureStorageDurableJobsExtensions.cs |
Azure Durable Jobs composition |
src/AWS/Orleans.Journaling.S3/S3JournalStorageHostingExtensions.cs |
Named S3 registration |
src/api/Redis/Orleans.Journaling.Redis/Orleans.Journaling.Redis.cs |
Redis public API |
src/api/Orleans.Journaling/Orleans.Journaling.cs |
Journaling public API |
src/api/Orleans.DurableJobs/Orleans.DurableJobs.cs |
Durable Jobs public API |
src/api/Azure/Orleans.Journaling.AzureStorage/Orleans.Journaling.AzureStorage.cs |
Azure journaling public API |
src/api/AWS/Orleans.Journaling.S3/Orleans.Journaling.S3.cs |
S3 public API |
samples/Samples.slnx |
Adds migration sample projects |
samples/README.md |
Updates sample catalog |
samples/gallery.json |
Adds sample metadata |
samples/DurableJobsMigration/README.md |
Migration sample usage |
samples/DurableJobsMigration/DurableJobsMigration/Program.cs |
Prepare/drain workflow |
samples/DurableJobsMigration/DurableJobsMigration/MigrationGrain.cs |
Sample scheduling grain |
samples/DurableJobsMigration/DurableJobsMigration/DurableJobsMigration.csproj |
Sample project configuration |
samples/DurableJobsMigration/DurableJobsMigration.AppHost/Properties/launchSettings.json |
Sample launch settings |
samples/DurableJobsMigration/DurableJobsMigration.AppHost/Program.cs |
Sample orchestration |
samples/DurableJobsMigration/DurableJobsMigration.AppHost/DurableJobsMigration.AppHost.csproj |
Sample host project |
samples/DurableJobsMigration/Directory.Packages.props |
Sample package versions |
playground/DurableJobsJournaling/README.md |
Playground migration instructions |
playground/DurableJobsJournaling/DurableJobsJournaling.Silo/StorageInventoryReporter.cs |
Inventory reporting |
playground/DurableJobsJournaling/DurableJobsJournaling.Silo/Program.cs |
Named provider configuration |
playground/DurableJobsJournaling/DurableJobsJournaling.AppHost/Program.cs |
Playground orchestration |
durable-jobs-provider-migration-plan.md |
Provider migration plan |
docs/site/src/content/docs/toc.yml |
Documentation navigation |
docs/site/src/content/docs/grains/journaling/redis-journal-storage.md |
Redis provider guidance |
docs/site/src/content/docs/grains/journaling/operations.md |
Provider-drain monitoring guidance |
docs/site/src/content/docs/grains/journaling/durable-jobs-migration.md |
Migration procedures |
docs/site/src/content/docs/grains/journaling/configuration.md |
Named provider configuration |
docs/site/src/content/docs/grains/journaling/azure-storage.md |
Azure provider documentation |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
ReubenBond
force-pushed
the
rb-feat-durable-jobs-journal-migration
branch
from
September 16, 2026 20:30
6e78801 to
d71519f
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Four unresolved findings remain, including two moderate sample issues, one moderate broken-link issue, and one nit.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
ReubenBond
force-pushed
the
rb-feat-durable-jobs-journal-migration
branch
from
September 16, 2026 20:49
d71519f to
ce38958
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved critical discovery-isolation and moderate inventory-consistency findings require fixes before approval.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (1)
ReubenBond
marked this pull request as ready for review
September 16, 2026 21:43
Contributor
Code coverage
Report-only conclusion: improved. The current-main baseline is commit Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
This was referenced Oct 3, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Durable Jobs needs to move new work to another journal storage namespace while existing jobs finish safely in their original namespace. Operators also need a complete inventory, including future and poisoned shards, to establish when a provider is ready for retirement.
Solution
Select the active named provider for new shards with
ActiveProviderName, alongside optional draining providers, through the common journaled Durable Jobs registration. New shards use the active provider; discovery combines selected catalogs with oldest-first ordering and a shared claim budget. Existing shards retain their original provider for replay, execution, retry, rescheduling, cancellation, and cleanup. Single-provider deployments retain direct metadata lookup, and provider-local failures remain isolated while caller cancellation propagates.Add a read-only full-namespace storage inspector, provider-migration guidance, and a runnable prepare/drain sample. Shard IDs and serialized job handles retain their existing shape. Durable Jobs provider migration is scoped independently from the Journaling and Messaging work in #10693.
Foundation
The named journal-provider foundation landed in #11275. This PR is rebased onto
main, and its diff contains the Durable Jobs migration changes.Microsoft Reviewers: Open in CodeFlow