You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Default-mapped Azure Table journal IDs support up to 512 characters, which encode to 1,024-character partition keys. Catalog prefix filters appended a sentinel beyond that limit, and longer range bounds also produced oversized PartitionKey literals.
Solution
Keep every default-mapping PartitionKey filter literal within 1,024 characters while preserving ordinal range semantics:
A 512-character prefix uses an inclusive upper comparison against its exact encoded key. Longer prefixes return an empty result before client access.
Longer bounds use their first 512 characters, with a strict lower comparison and an inclusive upper comparison. Existing unsupported-character boundary projection remains in effect.
Custom mappings retain their canonical JournalId property filters.
Regression coverage enforces literal lengths and exact native result sets for maximum-length prefixes, oversized bounds, unsupported characters around the cutoff, and long custom-mapped IDs. The provider README describes these query semantics.
Scope
The catalog foundation originally extracted from #11210 landed through #11258. After rebasing onto main, this PR contains only the remaining Azure Table key-limit correction, its focused regressions, and provider documentation.
Introduces a journaling catalog API with range-aware enumeration, metadata snapshots, and optimized provider-specific storage layouts and queries.
Changes:
Adds catalog entries, range options, and shared filtering logic.
Optimizes Volatile, Azure Blob/Table, Redis, and S3 enumeration.
Updates Durable Jobs integration, tests, documentation, and generated APIs.
Critical finding: New default layouts may make existing data undiscoverable after upgrades. Add dual-read compatibility or explicit migration/backfill guidance, especially for Redis.
The newest successful coverage run tested 4fa1669, not current main 437bb05.
Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.
The comparison remains report-only while normal line and branch variance is calibrated.
JournalId(string) accepts any non-blank UTF-16 value, including unpaired surrogates, but Uri.EscapeDataString throws for those values. Since GetJournalBaseKey is reached by CreateStorage, this layout change makes otherwise accepted journal ids fail before any Redis operation; use a reversible encoding which preserves UTF-16 code units (and update the decode path), or explicitly validate this input at the public boundary.
JournalId(string) permits raw UTF-16 values containing unpaired surrogates, but Uri.EscapeDataString(keyName) throws UriFormatException for those values. Since the constructor computes these keys immediately, a previously accepted journal ID now fails in CreateStorage, and the new catalog range code explicitly preserves such ordinal IDs. Either use a reversible encoding for arbitrary UTF-16 code units or validate and document a Redis-specific restriction before key construction.
Addressed the Redis raw UTF-16 review feedback in 413f898. Unicode scalars retain URI escaping; unpaired surrogates use reversible %uXXXX escapes, with literal percent signs escaped separately. Stored $journal-id values use the same encoding, preserving identities through Redis string transport and custom-mapped discovery. Canonical decoding rejects malformed keys explicitly.
The focused catalog and storage-transport tests passed on .NET 8 and .NET 10. Both real-Redis CI jobs also passed at 783d695, and their TRX reports confirm RawUtf16Identities_RoundTripThroughStorageAndCatalog passed for both default and custom mappings on both frameworks.
The branch is now rebased onto current main 8905026. That final synchronization changes only four upstream ADO.NET SQL files; the Redis implementation and tests are byte-identical to the CI-validated version. All five PR patches are unchanged in range-diff. Fresh checks for final head 413f898 are running.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Default-mapped Azure Table journal IDs support up to 512 characters, which encode to 1,024-character partition keys. Catalog prefix filters appended a sentinel beyond that limit, and longer range bounds also produced oversized PartitionKey literals.
Solution
Keep every default-mapping PartitionKey filter literal within 1,024 characters while preserving ordinal range semantics:
Regression coverage enforces literal lengths and exact native result sets for maximum-length prefixes, oversized bounds, unsupported characters around the cutoff, and long custom-mapped IDs. The provider README describes these query semantics.
Scope
The catalog foundation originally extracted from #11210 landed through #11258. After rebasing onto main, this PR contains only the remaining Azure Table key-limit correction, its focused regressions, and provider documentation.
Microsoft Reviewers: Open in CodeFlow