Repository navigation
feat(durable-jobs): discover shards by start time - #11258
Merged
ReubenBond merged 10 commits intoSep 15, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Two unresolved critical shard-discovery issues block approval.
Get a fresh assessment by requesting another Copilot review.
Review tier: Lite
Findings: 2
Open (2)
What changed in this PR
Adds time-addressable Durable Jobs shard discovery with bounded, oldest-first catalog scans, configurable timing, metadata/ETag ownership claims, and provider layout updates.
Changes:
- Adds timestamp-prefixed shard IDs and configurable discovery horizons.
- Updates Volatile, Azure, S3, and Redis catalog discovery.
- Adds streaming assignment, shutdown coordination, tests, documentation, and API updates.
| File | Reviewed changes |
|---|---|
test/Orleans.Journaling.Tests/VolatileJournalStorageProviderTests.cs |
Tests Volatile range discovery and metadata snapshots. |
test/Orleans.Journaling.Tests/S3JournalStorageTests.cs |
Tests S3 WAL listing behavior. |
test/Orleans.Journaling.Tests/JournalCatalogRangeTests.cs |
Tests catalog range semantics. |
test/Orleans.Journaling.Tests/AzureTableJournalStorageTests.cs |
Tests Azure Table storage changes. |
test/Orleans.Journaling.Tests/AzureTableJournalStorageProviderTests.cs |
Tests Azure Table provider behavior. |
test/Orleans.Journaling.Tests/AzureTableJournalStorageOptionsTests.cs |
Tests Table mapping options. |
test/Orleans.Journaling.Tests/AzureBlobJournalStorageTests.cs |
Tests Azure Blob storage behavior. |
test/Orleans.DurableJobs.Tests/DurableJobs/JournaledJobShardStateTests.cs |
Tests timestamp-based shard identities. |
test/Orleans.DurableJobs.Tests/DurableJobs/JournaledJobShardManagerTests.cs |
Tests shard manager behavior. |
test/Orleans.DurableJobs.Tests/DurableJobs/JournaledJobShardDiscoveryTests.cs |
Tests discovery ordering and conditional claims. |
test/Orleans.Core.Tests/DurableJobs/DurableJobsOptionsTests.cs |
Tests discovery option validation. |
test/Extensions/Orleans.Redis.Tests/Journaling/RedisJournalStorageTests.cs |
Tests Redis storage changes. |
test/Extensions/Orleans.Redis.Tests/Journaling/RedisJournalStorageCatalogTests.cs |
Tests Redis discovery mappings and scans. |
src/Redis/Orleans.Journaling.Redis/RedisJournalStorageProvider.cs |
Adds prefix-aware Redis catalog scanning. |
src/Redis/Orleans.Journaling.Redis/RedisJournalStorageOptions.cs |
Adds Redis discovery mapping options. |
src/Redis/Orleans.Journaling.Redis/RedisJournalStorage.cs |
Supports reversible discovery key names. |
src/Redis/Orleans.Journaling.Redis/README.md |
Documents Redis catalog behavior. |
src/Orleans.Journaling/VolatileJournalStorage.cs |
Implements indexed range discovery and snapshots. |
src/Orleans.Journaling/README.md |
Documents catalog behavior. Nit (1 vote): clarify default versus custom metadata-read costs. |
src/Orleans.Journaling/Orleans.Journaling.csproj |
Supports catalog changes. |
src/Orleans.Journaling/ListOptions.cs |
Adds range and metadata query options. |
src/Orleans.Journaling/JournalCatalogRange.cs |
Applies catalog prefixes and inclusive bounds. |
src/Orleans.Journaling/JournalCatalogEntry.cs |
Defines catalog entries with metadata snapshots. |
src/Orleans.Journaling/IJournalStorageCatalog.cs |
Updates catalog enumeration APIs. |
src/Orleans.DurableJobs/README.md |
Documents discovery configuration. |
src/Orleans.DurableJobs/LocalDurableJobManager.cs |
Integrates periodic discovery and shutdown draining. |
src/Orleans.DurableJobs/JournaledJobShardManager.cs |
Implements ordered bounded discovery and claims. Critical (2 votes): legacy random-named shards can be excluded by the timestamp bound; add compatibility discovery or migration. Critical (1 vote): cache replacement during unregister/reclaim can orphan an instance; use TryAdd and dispose the losing instance. |
src/Orleans.DurableJobs/JobShardManager.cs |
Adds streaming discovery support. |
src/Orleans.DurableJobs/JobShardId.cs |
Creates timestamp-prefixed shard identities. |
src/Orleans.DurableJobs/Hosting/DurableJobsOptions.cs |
Adds discovery horizon and check-interval options. |
src/Azure/Orleans.Journaling.AzureStorage/README.md |
Documents Azure catalog behavior. |
src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorageProvider.cs |
Adds indexed and canonical-ID catalog queries. |
src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorageOptions.cs |
Adds ordered default partition-key mapping. |
src/Azure/Orleans.Journaling.AzureStorage/AzureTableJournalStorage.cs |
Updates Table metadata handling. |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobJournalStorageProvider.cs |
Adds bounded Blob WAL-prefix listing. |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobJournalStorageOptions.cs |
Updates Blob naming configuration. |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobJournalStorageLayout.cs |
Defines Blob storage namespaces. |
src/Azure/Orleans.Journaling.AzureStorage/AzureBlobJournalStorage.cs |
Updates Blob WAL and checkpoint handling. |
src/AWS/Orleans.Journaling.S3/S3JournalStorageProvider.cs |
Adds bounded S3 catalog listing. |
src/AWS/Orleans.Journaling.S3/S3JournalStorageOptions.cs |
Defines S3 WAL/checkpoint layouts and mappings. |
src/AWS/Orleans.Journaling.S3/README.md |
Documents S3 catalog behavior. |
src/api/Orleans.Journaling/Orleans.Journaling.cs |
Regenerates the Journaling API surface. |
src/api/Orleans.DurableJobs/Orleans.DurableJobs.cs |
Regenerates the Durable Jobs API surface. |
src/api/AWS/Orleans.Journaling.S3/Orleans.Journaling.S3.cs |
Regenerates the S3 API surface. |
docs/site/src/content/docs/grains/journaling/azure-storage.md |
Documents the updated Azure Blob layout. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
Code coverage
Report-only conclusion: current-main baseline stale. The newest successful coverage run tested 5a63e15, not current main 4fa1669. Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities. The comparison remains report-only while normal line and branch variance is calibrated. Coverage details |
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The critical stale-snapshot ownership race must be fixed before approval.
Get a fresh assessment by requesting another Copilot review.
Review tier: Lite
Findings: 1
Open (1)
Resolved since last review (2)
Split the time-addressable discovery implementation from dotnet#11210, preserving the final source behavior and contributor fixes at 7c8dae0.
Preserve the existing cached shard when discovery races with unregister cleanup, await disposal of a losing instance, and retain claim budget accounting. Add a deterministic release/claim race regression covering cache identity, disposal, and subsequent job recovery.
Read current metadata when a projected local-owner snapshot has no cached shard, then apply the normal eligibility and conditional ownership decisions. Preserve cached instances and add deterministic coverage for release, deletion, reassignment, poisoned state, and cache reuse.
Assert inactive yielded shards are unregistered on shutdown and verify the descendant-only catalog prefix while cached projected shards require no storage opens.
ReubenBond
force-pushed
the
rb-feat-durable-jobs-due-discovery
branch
from
September 15, 2026 19:39
306cdb4 to
f5d1840
Compare
This was referenced Sep 15, 2026
This was referenced Oct 3, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Durable Jobs needs to recover overdue shards while limiting candidate discovery to a configurable near-term horizon, with predictable oldest-first assignment across catalog providers.
Solution
Name shard journals using fixed-width UTC start times and use bounded catalog queries for fresh discovery sweeps. Order and deduplicate candidates, stream assignments as shards open, and use metadata snapshots with ETags for conditional ownership updates. Configure the lookahead and check interval while preserving membership signals, claimed resources, and shutdown admission/drain guarantees.
This extracts the time-addressable discovery implementation and contributor fixes from #11210 at
7c8dae07d451cda3ebc949fc02ce23c15476d169. Follow-up review fixes retain the existing cached shard and dispose a duplicate claimed instance when discovery races with unregister cleanup. Projected local-owner entries read current metadata on cache misses before eligibility and ownership decisions. Discovery integrates the current shutdown behavior, including releasing inactive shards, and preserves the latest greenfield storage documentation.Dependencies and review
Depends on #11257 (catalog query and storage layouts), included at
783d695324aa468fd8d0243005bcbd199d6507b5. The shutdown prerequisite #11256 has landed and is included throughmain. The branch includesmainthrough8905026dc62e0d676e7e930ab702d0447e91d9a8.Review the isolated discovery slice. The comparison base combines the catalog prerequisite with current
main.The PR targets
dotnet/orleans:main, so its full diff currently includes the catalog prerequisite. After #11257 lands, rebase ontomainto present the standalone discovery diff, including when the prerequisite is squash-merged. Human review and sign-off govern merging.Microsoft Reviewers: Open in CodeFlow