Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Client encryption]: Refactor EncryptionProcessor #4757

Merged
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
d6f89ea
Add baseline benchmarks for `Microsoft.Azure.Cosmos.Encryption.Custom`
juraj-blazek Sep 12, 2024
8928b88
Cleanup
juraj-blazek Sep 12, 2024
9dc7d54
Use set of static test data for benchmarks
juraj-blazek Sep 12, 2024
368dc25
Merge branch 'master' into users/juraj-blazek/encryption-benchmark-ba…
juraj-blazek Sep 12, 2024
38a198a
Add non-allocating APIs to encryptors
juraj-blazek Sep 16, 2024
e0bb8bf
WIP
juraj-blazek Sep 16, 2024
9c3c276
Revert solution update
juraj-blazek Sep 16, 2024
c37a092
Merge branch 'users/juraj-blazek/encryption-benchmark-baseline' of ht…
juraj-blazek Sep 16, 2024
f4db1c6
Merge branch 'master' into users/juraj-blazek/encryption-benchmark-ba…
juraj-blazek Sep 16, 2024
91714f9
Merge branch 'master' into users/juraj-blazek/encryption-benchmark-ba…
juraj-blazek Sep 17, 2024
2058f8b
Merge remote-tracking branch 'origin/master' into users/juraj-blazek/…
juraj-blazek Sep 17, 2024
f33538c
Add implementation, fix tests
juraj-blazek Sep 17, 2024
6923fd2
Switch to randomized encryption for benchmarks
juraj-blazek Sep 17, 2024
4ea8690
Merge branch 'master' into users/juraj-blazek/encryption-benchmark-ba…
kr-santosh Sep 18, 2024
03ef682
Some more array pooling
juraj-blazek Sep 17, 2024
1044a89
Streaming deserialization
juraj-blazek Sep 19, 2024
f1db54d
Merge remote-tracking branch 'origin/master' into users/juraj-blazek/…
juraj-blazek Sep 19, 2024
05bfc50
Cleanup
juraj-blazek Sep 20, 2024
5629f74
Update MDE and rerun benchmarks
juraj-blazek Sep 20, 2024
495d2c4
Add non-allocating APIs to encryptors
juraj-blazek Sep 16, 2024
b3f74b1
Merge branch 'master' into users/juraj-blazek/non-allocating-encryptor
Sep 30, 2024
14bce37
~ drop repeated DEK calls
Sep 30, 2024
4ff1601
! typo
Sep 30, 2024
d8a345c
~ update benchmark
Sep 30, 2024
03c06e0
~ fix tests
Sep 30, 2024
a1025c4
Merge branch 'Azure:master' into users/juraj-blazek/non-allocating-en…
JanHyka Oct 1, 2024
3bf77c8
~ cleanup
Oct 1, 2024
ceaa8b5
+ refresh benchmark
Oct 1, 2024
611b3ac
+ unit test
Oct 1, 2024
9ca89d4
~ merge predecessor PR
Oct 1, 2024
8a78fe8
~ merge fixes and initial cleanup
Oct 1, 2024
8ed2135
~ write directly to output document instead of copying
Oct 1, 2024
bbe9845
! tests
Oct 1, 2024
a107f62
~ retrieve DataEncryptionKey only once per document
Oct 1, 2024
a1ad02b
! fix tests
Oct 1, 2024
4f2f072
~ update Aes algorithm to reuse GetEncryptedByteCount
Oct 2, 2024
cbbeee2
~ refactor EncryptionProcessor
Oct 4, 2024
b6c851c
! names
Oct 4, 2024
72ccae7
~ less static
Oct 4, 2024
8ea5879
Merge branch 'master' into users/juraj-blazek/encryption-array-pooling
Oct 6, 2024
8c60e69
Merge branch 'master' into users/juraj-blazek/encryption-array-pooling
Oct 6, 2024
5554aa0
~ merge fixes
Oct 6, 2024
28620ed
~ cleanup
Oct 6, 2024
eb059c8
~ unwanted changes
Oct 6, 2024
cc2eab5
- unused method
Oct 6, 2024
c9ba300
~ updates (PR)
Oct 6, 2024
9f9cbca
~ add stable vs preview release duplicity
Oct 7, 2024
c347e71
Merge branch 'users/juraj-blazek/encryption-array-pooling' into users…
Oct 7, 2024
64172b8
~ cleanup and parent branch merge
Oct 7, 2024
ff583f9
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
Oct 7, 2024
0ba8c34
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
Oct 7, 2024
326b1be
~ master merges
Oct 7, 2024
c520e16
- duplicate
Oct 7, 2024
5c40821
~ cleanup
Oct 7, 2024
535afd9
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
kirankumarkolli Oct 8, 2024
ea71a41
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
kirankumarkolli Oct 9, 2024
f11fefc
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
Oct 9, 2024
9cfc275
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
kirankumarkolli Oct 9, 2024
11e8090
~ fix build with SdkProjectRef and EncryptionPreview flags (hopefully)
Oct 9, 2024
47dfd18
Merge branch 'master' into users/jan-hyka/refactor-encryption-processor
kirankumarkolli Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Switch to randomized encryption for benchmarks
juraj-blazek committed Sep 17, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 6923fd2fdb29a831e6eadd01e26463c2e312d254
Original file line number Diff line number Diff line change
@@ -35,7 +35,7 @@ public async Task Setup()
Mock<DataEncryptionKeyProvider> keyProvider = new();
keyProvider
.Setup(x => x.FetchDataEncryptionKeyWithoutRawKeyAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(() => new MdeEncryptionAlgorithm(DekProperties, EncryptionType.Deterministic, StoreProvider.Object, cacheTimeToLive: TimeSpan.MaxValue));
.ReturnsAsync(() => new MdeEncryptionAlgorithm(DekProperties, EncryptionType.Randomized, StoreProvider.Object, cacheTimeToLive: TimeSpan.MaxValue));

this.encryptor = new(keyProvider.Object);
this.encryptionOptions = CreateEncryptionOptions();
Original file line number Diff line number Diff line change
@@ -8,12 +8,6 @@
<Nullable>enable</Nullable>
</PropertyGroup>

<ItemGroup>
<None Remove="sampledata\testdoc-100kb.json" />
<None Remove="sampledata\testdoc-10kb.json" />
<None Remove="sampledata\testdoc-1kb.json" />
</ItemGroup>

<ItemGroup>
<EmbeddedResource Include="sampledata\testdoc-100kb.json" />
<EmbeddedResource Include="sampledata\testdoc-10kb.json" />
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@

BenchmarkDotNet=v0.13.3, OS=Windows 11 (10.0.22631.4169)
11th Gen Intel Core i9-11950H 2.60GHz, 1 CPU, 16 logical and 8 physical cores
.NET SDK=9.0.100-preview.7.24407.12
.NET SDK=9.0.100-rc.1.24452.12
[Host] : .NET 6.0.33 (6.0.3324.36610), X64 RyuJIT AVX2

Job=MediumRun Toolchain=InProcessEmitToolchain IterationCount=15
@@ -11,9 +11,9 @@ LaunchCount=2 WarmupCount=10
```
| Method | DocumentSizeInKb | Mean | Error | StdDev | Gen0 | Gen1 | Gen2 | Allocated |
|-------- |----------------- |------------:|-----------:|-----------:|---------:|---------:|---------:|-----------:|
| **Encrypt** | **1** | **60.05 μs** | **1.537 μs** | **2.300 μs** | **5.0659** | **1.2817** | **-** | **62.65 KB** |
| Decrypt | 1 | 70.76 μs | 0.812 μs | 1.164 μs | 5.7373 | 1.4648 | - | 71.22 KB |
| **Encrypt** | **10** | **165.23 μs** | **3.741 μs** | **5.365 μs** | **21.2402** | **3.6621** | **-** | **262.38 KB** |
| Decrypt | 10 | 231.32 μs | 4.627 μs | 6.635 μs | 29.5410 | 3.4180 | - | 363.84 KB |
| **Encrypt** | **100** | **2,572.40 μs** | **242.163 μs** | **362.458 μs** | **201.1719** | **126.9531** | **125.0000** | **2466.27 KB** |
| Decrypt | 100 | 2,952.48 μs | 397.387 μs | 557.081 μs | 255.8594 | 210.9375 | 160.1563 | 3412.88 KB |
| **Encrypt** | **1** | **61.45 μs** | **1.676 μs** | **2.457 μs** | **4.9438** | **1.2207** | **-** | **61.25 KB** |
| Decrypt | 1 | 77.89 μs | 1.959 μs | 2.933 μs | 5.7373 | 1.4648 | - | 71.22 KB |
| **Encrypt** | **10** | **171.64 μs** | **3.341 μs** | **4.791 μs** | **21.2402** | **3.6621** | **-** | **260.97 KB** |
| Decrypt | 10 | 255.57 μs | 7.833 μs | 11.724 μs | 29.2969 | 4.3945 | - | 363.84 KB |
| **Encrypt** | **100** | **2,601.33 μs** | **215.481 μs** | **322.522 μs** | **199.2188** | **125.0000** | **123.0469** | **2464.88 KB** |
| Decrypt | 100 | 3,156.06 μs | 321.419 μs | 481.084 μs | 355.4688 | 300.7813 | 261.7188 | 3413.05 KB |