Skip to content

Routing: Fixes GetOverlappingRanges CPU overhead from repeated JSON deserialization#5748

Merged
kirankumarkolli merged 1 commit intomasterfrom
users/kirankk/copilot-5747-perf-getoverlappingranges
Apr 8, 2026
Merged

Routing: Fixes GetOverlappingRanges CPU overhead from repeated JSON deserialization#5748
kirankumarkolli merged 1 commit intomasterfrom
users/kirankk/copilot-5747-perf-getoverlappingranges

Conversation

@kirankumarkolli
Copy link
Copy Markdown
Member

@kirankumarkolli kirankumarkolli commented Apr 8, 2026 •
edited
Loading

🤖 This PR was generated with the help of GitHub Copilot CLI agent.

Motivation

Fixes #5747

CollectionRoutingMap.GetOverlappingRanges was spending excessive CPU time in JsonSerializable.GetValue<T> due to repeated access of PartitionKeyRange.MinInclusive. Each property access on PartitionKeyRange triggers JToken.ToObject<T>() because the base class JsonSerializable deserializes on every call with no caching.

A customer-reported PerfView trace showed ~14.8 seconds exclusive CPU in GetOverlappingRanges → GetValue out of a 21-minute trace window.

Changes

CollectionRoutingMap.cs:
Replaced this.OrderedPartitionKeyRanges[i].MinInclusive (triggers JSON deserialization on every access) with this.orderedRanges[i].Min (already-cached string materialized during construction).

CollectionRoutingMapBenchmark.cs (new):
Added BenchmarkDotNet benchmark for GetOverlappingRanges with 10/100/1000 partition counts.

Benchmark Results

Before (baseline on master)

Method PartitionCount Mean Allocated
GetOverlappingRanges_SingleRange 10 1,115 ns 288 B
GetOverlappingRanges_MultipleRanges 10 1,221 ns 256 B
GetOverlappingRanges_SingleRange 100 14,331 ns 1,328 B
GetOverlappingRanges_MultipleRanges 100 24,717 ns 2,368 B
GetOverlappingRanges_SingleRange 1000 218,005 ns 8,640 B
GetOverlappingRanges_MultipleRanges 1000 346,477 ns 16,848 B

After (this PR)

Method PartitionCount Mean Allocated
GetOverlappingRanges_SingleRange 10 687 ns 288 B
GetOverlappingRanges_MultipleRanges 10 809 ns 256 B
GetOverlappingRanges_SingleRange 100 10,746 ns 1,328 B
GetOverlappingRanges_MultipleRanges 100 19,285 ns 2,368 B
GetOverlappingRanges_SingleRange 1000 173,112 ns 8,640 B
GetOverlappingRanges_MultipleRanges 1000 288,649 ns 16,848 B

Improvement Summary

Method 10 partitions 100 partitions 1000 partitions
SingleRange 1.6x faster 1.3x faster 1.3x faster
MultipleRanges 1.5x faster 1.3x faster 1.2x faster

Note: Further optimization by replacing the per-call SortedList<string, PartitionKeyRange> allocation with index-based dedup yields an additional ~13x improvement (17x total vs baseline). This is tracked as a follow-up since the partition ranges are contiguous and sorted by construction, making the SortedList dedup unnecessary.

Testing

  • All 8 existing CollectionRoutingMapTest unit tests pass (default + preview builds)
  • New CollectionRoutingMapBenchmark validates performance characteristics

🤖 This PR was generated with the help of GitHub Copilot CLI agent.

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Apr 8, 2026

Azure Pipelines:
Successfully started running 1 pipeline(s).

kirankumarkolli
kirankumarkolli commented Apr 8, 2026
View reviewed changes
Comment thread Microsoft.Azure.Cosmos/src/Routing/CollectionRoutingMap.cs Outdated
@kirankumarkolli kirankumarkolli force-pushed the users/kirankk/copilot-5747-perf-getoverlappingranges branch from 21b66ae to 10ba8aa Compare April 8, 2026 14:01
@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Apr 8, 2026

Azure Pipelines:
Successfully started running 1 pipeline(s).

kirankumarkolli
kirankumarkolli commented Apr 8, 2026
View reviewed changes
Comment thread Microsoft.Azure.Cosmos/src/Routing/CollectionRoutingMap.cs Outdated
@kirankumarkolli
Routing: Fixes GetOverlappingRanges CPU overhead from repeated JSON d…
8c5cfcd 
…eserialization

Eliminates repeated JsonSerializable.GetValue calls in the hot path of
CollectionRoutingMap.GetOverlappingRanges by using already-cached
orderedRanges[i].Min instead of OrderedPartitionKeyRanges[i].MinInclusive.

Also removes per-call SortedList<string, PartitionKeyRange> allocation by
using index-based dedup with the contiguous ordered ranges.

Benchmark results (1000 partitions):
- SingleRange:   218,005 ns -> 12,743 ns  (17x faster, 75% less allocation)
- MultipleRanges: 346,477 ns -> 20,194 ns (17x faster, 70% less allocation)

Fixes #5747

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kirankumarkolli kirankumarkolli force-pushed the users/kirankk/copilot-5747-perf-getoverlappingranges branch from 10ba8aa to 8c5cfcd Compare April 8, 2026 14:11
@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Apr 8, 2026

Azure Pipelines:
Successfully started running 1 pipeline(s).

@kirankumarkolli kirankumarkolli marked this pull request as ready for review April 8, 2026 14:14
@azure-pipelines
Copy link
Copy Markdown

azure-pipelines Bot commented Apr 8, 2026

Azure Pipelines:
Successfully started running 1 pipeline(s).

FabianMeiswinkel
FabianMeiswinkel approved these changes Apr 8, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@FabianMeiswinkel FabianMeiswinkel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

kundadebdatta
kundadebdatta approved these changes Apr 8, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@kundadebdatta kundadebdatta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@kirankumarkolli kirankumarkolli merged commit da5dcbf into master Apr 8, 2026
35 checks passed
@kirankumarkolli kirankumarkolli deleted the users/kirankk/copilot-5747-perf-getoverlappingranges branch April 8, 2026 22:54
@NaluTripician NaluTripician mentioned this pull request Apr 24, 2026
Merged
4 tasks
microsoft-github-policy-service Bot pushed a commit that referenced this pull request Apr 25, 2026
@NaluTripician
Release: Adds version 3.59.0 (#5810)
dd06aae 
## Release 3.59.0

### Version Changes
- ClientOfficialVersion: 3.58.0 → 3.59.0
- ClientPreviewVersion: 3.59.0 → 3.60.0
- ClientPreviewSuffixVersion: preview.0 → preview.0

### Changelog (3.59.0 GA)

#### Added
- [5579](#5579)
Change Feed Processor: Adds Lease container export support
- [5709](#5709)
Performance: Adds caching for URL-encoded AAD authorization signature
- [5731](#5731) DNS
dot-suffix: Adds TCP DNS dot-suffix for Direct mode to avoid Kubernetes
ndots latency
- [5755](#5755)
Exceptionless: Adds enabling exception less 400 status code
- [5756](#5756)
Exceptionless: Adds enabling exception less 404/1002 status code
- [5757](#5757)
Exceptionless: Adds enabling exception less 403
- [5779](#5779)
Direct: Adds Direct package version bump to 3.42.4
- [5786](#5786)
Region Availability: Adds missing regions from Direct 3.42.4
- [5788](#5788)
Socket Handler: Adds HTTP/2 PING keep-alive to detect broken connections
in pool

#### Fixed
- [5553](#5553)
NativeDLLs: Fixes Conditionally include win-x64 native DLLs based on
RuntimeIdentifier
- [5588](#5588)
LINQ: Fixes memory leak from Expression.Compile() in all call sites
- [5617](#5617)
ChangeFeedProcessor: Fixes first-change skip during initial startup by
anchoring StartTime
- [5636](#5636)
CosmosClientBuilder: Fixes self-referencing loop in
GetSerializedConfiguration with STJ TypeInfoResolver
- [5748](#5748)
Routing: Fixes GetOverlappingRanges CPU overhead from repeated JSON
deserialization
- [5807](#5807)
ChangeFeedProcessor: Fixes lease de-duplication for
/partitionKey-partitioned lease containers

### Changelog (3.60.0-preview.0)

#### Added
- [5804](#5804)
SemanticReranking: Adds Configurable Request Timeout

#### Fixed
- [5783](#5783)
Container: Fixes SemanticRerankAsync TypeLoadException in derived
classes

### API Contract Diff (GA)
```diff
diff --git "a/Microsoft.Azure.Cosmos\\contracts\\API_3.58.0.txt" "b/Microsoft.Azure.Cosmos\\contracts\\API_3.59.0.txt"
index 1b74a69..6fa9352 100644
--- "a/Microsoft.Azure.Cosmos\\contracts\\API_3.58.0.txt"
+++ "b/Microsoft.Azure.Cosmos\\contracts\\API_3.59.0.txt"
@@ -60,6 +60,7 @@ namespace Microsoft.Azure.Cosmos
         public ChangeFeedProcessor Build();
         public ChangeFeedProcessorBuilder WithErrorNotification(Container.ChangeFeedMonitorErrorDelegate errorDelegate);
         public virtual ChangeFeedProcessorBuilder WithInMemoryLeaseContainer();
+        public virtual ChangeFeedProcessorBuilder WithInMemoryLeaseContainer(MemoryStream leaseState);
         public ChangeFeedProcessorBuilder WithInstanceName(string instanceName);
         public ChangeFeedProcessorBuilder WithLeaseAcquireNotification(Container.ChangeFeedMonitorLeaseAcquireDelegate acquireDelegate);
         public ChangeFeedProcessorBuilder WithLeaseConfiguration(Nullable<TimeSpan> acquireInterval=default(Nullable<TimeSpan>), Nullable<TimeSpan> expirationInterval=default(Nullable<TimeSpan>), Nullable<TimeSpan> renewInterval=default(Nullable<TimeSpan>));
@@ -956,6 +957,7 @@ namespace Microsoft.Azure.Cosmos
         public const string NorwayWest = "Norway West";
         public const string PolandCentral = "Poland Central";
         public const string QatarCentral = "Qatar Central";
+        public const string SaudiArabiaEast = "Saudi Arabia East";
         public const string SingaporeCentral = "Singapore Central";
         public const string SingaporeNorth = "Singapore North";
         public const string SouthAfricaNorth = "South Africa North";
@@ -963,6 +965,7 @@ namespace Microsoft.Azure.Cosmos
         public const string SouthCentralUS = "South Central US";
         public const string SouthCentralUS2 = "South Central US 2";
         public const string SoutheastAsia = "Southeast Asia";
+        public const string SoutheastAsia3 = "Southeast Asia 3";
         public const string SoutheastUS = "Southeast US";
         public const string SoutheastUS3 = "Southeast US 3";
         public const string SoutheastUS5 = "Southeast US 5";
@@ -990,6 +993,7 @@ namespace Microsoft.Azure.Cosmos
         public const string USSecWest = "USSec West";
         public const string USSecWestCentral = "USSec West Central";
         public const string WestCentralUS = "West Central US";
+        public const string WestCentralUSFRE = "West Central US FRE";
         public const string WestEurope = "West Europe";
         public const string WestIndia = "West India";
         public const string WestUS = "West US";

```

### API Contract Diff (Preview)
```diff
diff --git "a/Microsoft.Azure.Cosmos\\contracts\\API_3.59.0-preview.0.txt" "b/Microsoft.Azure.Cosmos\\contracts\\API_3.60.0-preview.0.txt"
index 1ae52c0..58df10f 100644
--- "a/Microsoft.Azure.Cosmos\\contracts\\API_3.59.0-preview.0.txt"
+++ "b/Microsoft.Azure.Cosmos\\contracts\\API_3.60.0-preview.0.txt"
@@ -91,6 +91,7 @@ namespace Microsoft.Azure.Cosmos
         public ChangeFeedProcessor Build();
         public ChangeFeedProcessorBuilder WithErrorNotification(Container.ChangeFeedMonitorErrorDelegate errorDelegate);
         public virtual ChangeFeedProcessorBuilder WithInMemoryLeaseContainer();
+        public virtual ChangeFeedProcessorBuilder WithInMemoryLeaseContainer(MemoryStream leaseState);
         public ChangeFeedProcessorBuilder WithInstanceName(string instanceName);
         public ChangeFeedProcessorBuilder WithLeaseAcquireNotification(Container.ChangeFeedMonitorLeaseAcquireDelegate acquireDelegate);
         public ChangeFeedProcessorBuilder WithLeaseConfiguration(Nullable<TimeSpan> acquireInterval=default(Nullable<TimeSpan>), Nullable<TimeSpan> expirationInterval=default(Nullable<TimeSpan>), Nullable<TimeSpan> renewInterval=default(Nullable<TimeSpan>));
@@ -302,7 +303,7 @@ namespace Microsoft.Azure.Cosmos
         public abstract Task<ResponseMessage> ReplaceItemStreamAsync(Stream streamPayload, string id, PartitionKey partitionKey, ItemRequestOptions requestOptions=null, CancellationToken cancellationToken=default(CancellationToken));
         public abstract Task<ThroughputResponse> ReplaceThroughputAsync(ThroughputProperties throughputProperties, RequestOptions requestOptions=null, CancellationToken cancellationToken=default(CancellationToken));
         public abstract Task<ThroughputResponse> ReplaceThroughputAsync(int throughput, RequestOptions requestOptions=null, CancellationToken cancellationToken=default(CancellationToken));
-        public abstract Task<SemanticRerankResult> SemanticRerankAsync(string rerankContext, IEnumerable<string> documents, IDictionary<string, object> options=null, CancellationToken cancellationToken=default(CancellationToken));
+        public virtual Task<SemanticRerankResult> SemanticRerankAsync(string rerankContext, IEnumerable<string> documents, IDictionary<string, object> options=null, CancellationToken cancellationToken=default(CancellationToken));
         public abstract Task<ItemResponse<T>> UpsertItemAsync<T>(T item, Nullable<PartitionKey> partitionKey=default(Nullable<PartitionKey>), ItemRequestOptions requestOptions=null, CancellationToken cancellationToken=default(CancellationToken));
         public abstract Task<ResponseMessage> UpsertItemStreamAsync(Stream streamPayload, PartitionKey partitionKey, ItemRequestOptions requestOptions=null, CancellationToken cancellationToken=default(CancellationToken));
         public delegate Task ChangeFeedHandlerWithManualCheckpoint<T>(ChangeFeedProcessorContext context, IReadOnlyCollection<T> changes, Func<Task> checkpointAsync, CancellationToken cancellationToken);
@@ -407,6 +408,7 @@ namespace Microsoft.Azure.Cosmos
         public int GatewayModeMaxConnectionLimit { get; set; }
         public Func<HttpClient> HttpClientFactory { get; set; }
         public Nullable<TimeSpan> IdleTcpConnectionTimeout { get; set; }
+        public TimeSpan InferenceRequestTimeout { get; set; }
         public bool LimitToEndpoint { get; set; }
         public Nullable<int> MaxRequestsPerTcpConnection { get; set; }
         public Nullable<int> MaxRetryAttemptsOnRateLimitedRequests { get; set; }
@@ -1092,6 +1094,7 @@ namespace Microsoft.Azure.Cosmos
         public const string NorwayWest = "Norway West";
         public const string PolandCentral = "Poland Central";
         public const string QatarCentral = "Qatar Central";
+        public const string SaudiArabiaEast = "Saudi Arabia East";
         public const string SingaporeCentral = "Singapore Central";
         public const string SingaporeNorth = "Singapore North";
         public const string SouthAfricaNorth = "South Africa North";
@@ -1099,6 +1102,7 @@ namespace Microsoft.Azure.Cosmos
         public const string SouthCentralUS = "South Central US";
         public const string SouthCentralUS2 = "South Central US 2";
         public const string SoutheastAsia = "Southeast Asia";
+        public const string SoutheastAsia3 = "Southeast Asia 3";
         public const string SoutheastUS = "Southeast US";
         public const string SoutheastUS3 = "Southeast US 3";
         public const string SoutheastUS5 = "Southeast US 5";
@@ -1126,6 +1130,7 @@ namespace Microsoft.Azure.Cosmos
         public const string USSecWest = "USSec West";
         public const string USSecWestCentral = "USSec West Central";
         public const string WestCentralUS = "West Central US";
+        public const string WestCentralUSFRE = "West Central US FRE";
         public const string WestEurope = "West Europe";
         public const string WestIndia = "West India";
         public const string WestUS = "West US";
@@ -1504,6 +1509,7 @@ namespace Microsoft.Azure.Cosmos.Fluent
         public CosmosClientBuilder WithEnableRemoteRegionPreferredForSessionRetry(bool enableRemoteRegionPreferredForSessionRetry);
         public CosmosClientBuilder WithFaultInjection(IFaultInjector faultInjector);
         public CosmosClientBuilder WithHttpClientFactory(Func<HttpClient> httpClientFactory);
+        public CosmosClientBuilder WithInferenceRequestTimeout(TimeSpan inferenceRequestTimeout);
         public CosmosClientBuilder WithLimitToEndpoint(bool limitToEndpoint);
         public CosmosClientBuilder WithPriorityLevel(PriorityLevel priorityLevel);
         public CosmosClientBuilder WithReadConsistencyStrategy(ReadConsistencyStrategy readConsistencyStrategy);

```

### Checklist
- [ ] Changelog entries reviewed by team
- [ ] API contract diff reviewed by Kiran and Kirill
- [ ] Preview APIs reviewed (email sent to
azurecosmossdkdotnet@microsoft.com)
- [ ] Kiran sign-off obtained

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Apr 25, 2026
Merged
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@FabianMeiswinkel FabianMeiswinkel FabianMeiswinkel approved these changes

@kundadebdatta kundadebdatta kundadebdatta approved these changes

@khdang khdang Awaiting requested review from khdang khdang is a code owner

@sboshra sboshra Awaiting requested review from sboshra sboshra is a code owner

@adityasa adityasa Awaiting requested review from adityasa adityasa is a code owner

@neildsh neildsh Awaiting requested review from neildsh neildsh is a code owner

@kirillg kirillg Awaiting requested review from kirillg kirillg is a code owner

@Pilchie Pilchie Awaiting requested review from Pilchie Pilchie is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Performance: CollectionRoutingMap.GetOverlappingRanges spends excessive CPU in JsonSerializable.GetValue

3 participants

@kirankumarkolli @FabianMeiswinkel @kundadebdatta