From ed0e223a4bc1da295e2645c4a0c3a0f5a52541fd Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Tue, 18 Jul 2023 19:31:53 -0700 Subject: [PATCH 1/8] Draft of local-quorum documentation --- docs/LocalQuorum.md | 54 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 docs/LocalQuorum.md diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md new file mode 100644 index 0000000000..78cb84416c --- /dev/null +++ b/docs/LocalQuorum.md @@ -0,0 +1,54 @@ +> # INTERNAL AND DEVELOPMENT USAGE ONLY +> # NOT MEANT FOR PRODUCTION USAGE + + +## Context +Distributed databases that rely on replication for high availability, low latency, or both, must make a fundamental tradeoff between the read consistency, availability, latency, and throughput. + +The linearizability of the strong consistency model is the gold standard of data programmability. But it adds a steep price from higher write latencies due to data having to replicate and commit across large distances. Strong consistency may also suffer from reduced availability (during failures) because data can't replicate and commit in every region. Eventual consistency offers higher availability and better performance, but it's more difficult to program applications because data may not be consistent across all regions + + +Please refer to [public documentation](https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels) for more details. + + +Many applications can benefit from having a read-write model like below + +| Operation | Write-Region | Replicated-Regions | +|---|---|---| +|Write | Guaranteed write completion in write region | Asynchronous non-blocking replication | +|Read | Quorum read (read my writes) | Quorum read (eventual reads) | + + + +## How-TO +It involves two stages + +SDK version: MA [3.35.1](https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/changelog.md#-3351---2023-06-27) or [minimum recommended version](https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/changelog.md#-recommended-version) + +1. Enabling/opt-in ability to upgrade consistency level +```C# +CosmosClientOptions clientOptions = new CosmosClientOptions(); +var upgradeConsistencyProperty = clientOptions.GetType().GetProperty("EnableUpgradeConsistencyToLocalQuorum", BindingFlags.NonPublic | BindingFlags.Instance); +upgradeConsistencyProperty.SetValue(clientOptions, true); + +CosmosClient cosmosClient = new CosmosClient(..., clientOptions); +``` + +2. Per request upgrade consistency to Bounded +```C# +ItemRequestOptions requestOption = new ItemRequestOptions(); +requestOption.ConsistencyLevel = ConsistencyLevel.Bounded; + +T item = await container.ReadItemAsync(docId, new PartitionKey(docPartitionKey), requestOption); +``` + +```C# +QueryRequestOptions requestOption = new QueryRequestOptions(); +requestOption.ConsistencyLevel = ConsistencyLevel.Bounded; + +await container.GetItemQueryIterator(queryText, continuationToken, requestOption); +``` + + +> #### Please use Bounded only for per request options as pattern +> #### Single master account possibly Strong == Bounded (**TBD**) \ No newline at end of file From d2d6db70815227d345038744c46b040772d124ac Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Tue, 18 Jul 2023 19:34:02 -0700 Subject: [PATCH 2/8] Adding experimental to header --- docs/LocalQuorum.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index 78cb84416c..3e2fa7f24c 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -1,5 +1,4 @@ -> # INTERNAL AND DEVELOPMENT USAGE ONLY -> # NOT MEANT FOR PRODUCTION USAGE +> # INTERNAL AND DEVELOPMENT(EXPERIMENTAL) USAGE ONLY ## Context From 31ca8a99dbd97810b76e3f70008aad4c98b46ba9 Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Tue, 18 Jul 2023 19:36:56 -0700 Subject: [PATCH 3/8] Adding cross-region read guarantees --- docs/LocalQuorum.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index 3e2fa7f24c..6f438d662c 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -15,8 +15,9 @@ Many applications can benefit from having a read-write model like below | Operation | Write-Region | Replicated-Regions | |---|---|---| |Write | Guaranteed write completion in write region | Asynchronous non-blocking replication | -|Read | Quorum read (read my writes) | Quorum read (eventual reads) | +|Read | Quorum read (read my writes)
Monotonic reads in the region | Quorum read (eventual reads)
Monotonic reads in the region | +> ### NOTE: cross-region reads will violate the monotonic reads guarantee ## How-TO From 60d42732aa29417e0648d70a0498a97bb5481f7a Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Tue, 18 Jul 2023 19:39:44 -0700 Subject: [PATCH 4/8] Reads Bounded clarification --- docs/LocalQuorum.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index 6f438d662c..a7eadd0279 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -35,6 +35,9 @@ CosmosClient cosmosClient = new CosmosClient(..., clientOptions); ``` 2. Per request upgrade consistency to Bounded +> #### Please note that Bounded here is only used as HINT for SDK to do quorum reads +> #### It will not impact CosmosDB account or write consistency levels + ```C# ItemRequestOptions requestOption = new ItemRequestOptions(); requestOption.ConsistencyLevel = ConsistencyLevel.Bounded; @@ -49,6 +52,5 @@ requestOption.ConsistencyLevel = ConsistencyLevel.Bounded; await container.GetItemQueryIterator(queryText, continuationToken, requestOption); ``` - > #### Please use Bounded only for per request options as pattern > #### Single master account possibly Strong == Bounded (**TBD**) \ No newline at end of file From af136e506e34cb3b49cf45ec444e52f225eaabb8 Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Tue, 18 Jul 2023 19:49:44 -0700 Subject: [PATCH 5/8] Adding account consistency step also --- docs/LocalQuorum.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index a7eadd0279..11d43a7063 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -21,11 +21,13 @@ Many applications can benefit from having a read-write model like below ## How-TO -It involves two stages +It involves three stages +#### Create CosmosDB account with Eventual/ConsistentPrefix/Session consistency + +#### Enabling/opt-in ability to upgrade consistency level SDK version: MA [3.35.1](https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/changelog.md#-3351---2023-06-27) or [minimum recommended version](https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/changelog.md#-recommended-version) -1. Enabling/opt-in ability to upgrade consistency level ```C# CosmosClientOptions clientOptions = new CosmosClientOptions(); var upgradeConsistencyProperty = clientOptions.GetType().GetProperty("EnableUpgradeConsistencyToLocalQuorum", BindingFlags.NonPublic | BindingFlags.Instance); @@ -34,9 +36,9 @@ upgradeConsistencyProperty.SetValue(clientOptions, true); CosmosClient cosmosClient = new CosmosClient(..., clientOptions); ``` -2. Per request upgrade consistency to Bounded -> #### Please note that Bounded here is only used as HINT for SDK to do quorum reads -> #### It will not impact CosmosDB account or write consistency levels +#### Per request upgrade consistency to Bounded +> ###### Please note that Bounded here is only used as HINT for SDK to do quorum reads +> ###### It will not impact CosmosDB account or write consistency levels ```C# ItemRequestOptions requestOption = new ItemRequestOptions(); From c1d98684e33d46ae9c1fe0e9dd72561b6443e8df Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Wed, 19 Jul 2023 05:31:00 -0700 Subject: [PATCH 6/8] Non-Prod usage note at top --- docs/LocalQuorum.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index 11d43a7063..a2996baa92 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -1,4 +1,5 @@ -> # INTERNAL AND DEVELOPMENT(EXPERIMENTAL) USAGE ONLY +> # NOT SUPPORTED FOR PRODUCTION USAGE +> # ONLY INTERNAL, DEVELOPMENT OR EXPERIMENTAL USAGE ONLY ## Context From 69fae5e7e6333b05570e714aa3a9872419bc0a53 Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Wed, 19 Jul 2023 09:07:11 -0700 Subject: [PATCH 7/8] Addressing review comments --- docs/LocalQuorum.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index a2996baa92..07f2aeb437 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -5,7 +5,7 @@ ## Context Distributed databases that rely on replication for high availability, low latency, or both, must make a fundamental tradeoff between the read consistency, availability, latency, and throughput. -The linearizability of the strong consistency model is the gold standard of data programmability. But it adds a steep price from higher write latencies due to data having to replicate and commit across large distances. Strong consistency may also suffer from reduced availability (during failures) because data can't replicate and commit in every region. Eventual consistency offers higher availability and better performance, but it's more difficult to program applications because data may not be consistent across all regions +The linearizability of the strong consistency model is the gold standard of data programmability. But it adds a steep price from higher write latencies due to data having to replicate and commit across large distances. Strong consistency may also suffer from reduced availability (during failures) because data can't replicate and commit in every region. Eventual consistency offers higher availability and better performance, but it's more difficult to program applications because data may not be consistent across all regions (eventual reads are not guaranteed to be monotonic). Please refer to [public documentation](https://learn.microsoft.com/en-us/azure/cosmos-db/consistency-levels) for more details. @@ -15,8 +15,8 @@ Many applications can benefit from having a read-write model like below | Operation | Write-Region | Replicated-Regions | |---|---|---| -|Write | Guaranteed write completion in write region | Asynchronous non-blocking replication | -|Read | Quorum read (read my writes)
Monotonic reads in the region | Quorum read (eventual reads)
Monotonic reads in the region | +|Write | Guaranteed write completion in write region
vs Bounded: write un-availability when bounds are violated | Asynchronous non-blocking replication (eventual)
vs Bounded: Limits staleness by bounds | +|Read |**Single write region**: Read my writes
**Multiple write regions**: read-my-writes from the region of writes, otherwise eventual
Monotonic reads
No sessionToken management| **Single write region**: Eventual read
**Multiple write regions**: read-my-writes from the region of writes, otherwise eventual
Monotonic reads
No sessionToken management | > ### NOTE: cross-region reads will violate the monotonic reads guarantee From eadd69557b3e1a877601209fe089a5af0f1c94c0 Mon Sep 17 00:00:00 2001 From: Kiran Kumar Kolli Date: Wed, 19 Jul 2023 09:11:02 -0700 Subject: [PATCH 8/8] Some more refinement --- docs/LocalQuorum.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/LocalQuorum.md b/docs/LocalQuorum.md index 07f2aeb437..5ca45e9055 100644 --- a/docs/LocalQuorum.md +++ b/docs/LocalQuorum.md @@ -15,7 +15,7 @@ Many applications can benefit from having a read-write model like below | Operation | Write-Region | Replicated-Regions | |---|---|---| -|Write | Guaranteed write completion in write region
vs Bounded: write un-availability when bounds are violated | Asynchronous non-blocking replication (eventual)
vs Bounded: Limits staleness by bounds | +|Write | Write high availabillity in write region
vs BoundedStaleness: write un-availability when bounds are violated | Eventual replication: Asynchronous non-blocking, possibly unbounded staleness
vs BoundedStaleness: staleness limited by bounds | |Read |**Single write region**: Read my writes
**Multiple write regions**: read-my-writes from the region of writes, otherwise eventual
Monotonic reads
No sessionToken management| **Single write region**: Eventual read
**Multiple write regions**: read-my-writes from the region of writes, otherwise eventual
Monotonic reads
No sessionToken management | > ### NOTE: cross-region reads will violate the monotonic reads guarantee