Add Dapr.Cryptography package + fix for large files #1527

WhitWaldo · 2025-05-01T16:04:02Z

Description

This PR implements three things:

It splits out the cryptography building block to a separate Dapr.Cryptography package to be released with 1.16. All existing methods on Dapr.Client have an updated [Obsolete] attribute with a message telling developers that this method will be removed with the release of 1.17.
An issue was reported as a PR which identified a race condition wherein either the read or write buffer could run out of content and the other stream would stall as they were filling and draining in tandem. This PR fundamentally rewrites the implementation so this is no longer the case. A separate PR will be created to fix the implementation in the DaprClient so it can be patched in a 1.15 release.
I've updated the cryptography example to use the separate Dapr.Cryptography package and modernized to reflect a DI-first approach. It features three samples that demonstrate encryption and decryption of strings, streams and byte arrays of small (a dozen bytes), medium (~1 KB) and 1 GB files.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: #1488 dapr/dapr#8244

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Extended the documentation

Signed-off-by: Whit Waldo <[email protected]>

…tory Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

…equire more than one message to process Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

… for in a separate file Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/_index.md

daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/dotnet-cryptography-usage.md

examples/Cryptography/Examples/EncryptDecryptLargeFileExample.cs

Signed-off-by: Whit Waldo <[email protected]>

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

…-cryptography-usage.md Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

watfordsuzy

Couple of nits, some notes, and an unobserved task exception issue.

examples/Cryptography/Examples/EncryptDecryptLargeFileExample.cs

src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs

src/Dapr.Cryptography/Encryption/EncryptionStreamProcessor.cs

src/Dapr.Cryptography/Encryption/Models/DecryptionOptions.cs

watfordsuzy · 2025-05-01T20:10:34Z

src/Dapr.Cryptography/Encryption/DaprEncryptionGrpcClient.cs

+        var duplexStream = Client.DecryptAlpha1(grpcCallOptions);
+
+        using var streamProcessor = new DecryptionStreamProcessor();
+        await streamProcessor.ProcessStreamAsync(ciphertextStream, duplexStream, options.StreamingBlockSizeInBytes,


This await doesn't really do anything given how the processor is setup.

I think a design like you see in the Azure.Data.Tables library would be amenable here: https://learn.microsoft.com/en-us/dotnet/api/azure.data.tables.tableclient.queryasync?view=azure-dotnet

Basically, instead of returning a Task the process stream returns a wrapper which includes the IAsyncEnumerable.

It's true that await doesn't really do anything, but between it and just discarding the value, C# requires that I put something there. I don't know that I want to return any additional public types just for the sake of it when I could just return the IAsyncEnumerable and leave it to consumers to decide what to do with it.

watfordsuzy · 2025-05-01T20:16:08Z

src/Dapr.Cryptography/Encryption/DaprEncryptionGrpcClient.cs

+        var duplexStream = Client.EncryptAlpha1(grpcCallOptions);
+
+        using var streamProcessor = new EncryptionStreamProcessor();
+        await streamProcessor.ProcessStreamAsync(plaintextStream, duplexStream, encryptRequestOptions,


One interesting thing I noticed in the dapr Go code is that it does encryption/decryption on the full buffer in memory (byte[]) rather than streaming.

I can't speak to what happens in the runtime, but the specification is that this is implemented using a bidirectional streaming connection.

Signed-off-by: Whit Waldo <[email protected]>

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

…wing Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

…uggestion. Signed-off-by: Whit Waldo <[email protected]>

…ly block in case of exception Signed-off-by: Whit Waldo <[email protected]>

… the scope was no longer effective. Signed-off-by: Whit Waldo <[email protected]>

Signed-off-by: Whit Waldo <[email protected]>

* Implementation of the new crypto client Signed-off-by: Whit Waldo <[email protected]> Co-authored-by: Christopher Watford <[email protected]>

WhitWaldo added 8 commits April 20, 2025 20:43

Initial implementation of the new crypto client

c320849

Signed-off-by: Whit Waldo <[email protected]>

Re-added the streaming subscription example project

55a8a3d

Signed-off-by: Whit Waldo <[email protected]>

Re-added streaming example to a different project under another direc…

d311f92

…tory Signed-off-by: Whit Waldo <[email protected]>

Updated example to use new crypto library

ff60d70

Signed-off-by: Whit Waldo <[email protected]>

Tentative fix for Dapr crypto in separate client

97557b4

Signed-off-by: Whit Waldo <[email protected]>

Fixed bug in crypto implementation preventing files large enough to r…

9022458

…equire more than one message to process Signed-off-by: Whit Waldo <[email protected]>

Modernized crypto example

c73e5da

Signed-off-by: Whit Waldo <[email protected]>

Refactoring to rename from Dapr.Crypto to Dapr.Cryptography

0828ac6

Signed-off-by: Whit Waldo <[email protected]>

WhitWaldo added the area/client/crypto label May 1, 2025

WhitWaldo self-assigned this May 1, 2025

WhitWaldo requested review from a team as code owners May 1, 2025 16:04

WhitWaldo added 15 commits May 1, 2025 11:11

Fixed typo

34d27ef

Signed-off-by: Whit Waldo <[email protected]>

Removed empty section

a4b52ef

Signed-off-by: Whit Waldo <[email protected]>

Refactored cryptography example to move out of the Client folder

18aea94

Signed-off-by: Whit Waldo <[email protected]>

Added documentation for Dapr.Cryptography

f6ab29b

Signed-off-by: Whit Waldo <[email protected]>

Fixed package references

0b59a1a

Signed-off-by: Whit Waldo <[email protected]>

Fixed project import in solution file

3df9170

Signed-off-by: Whit Waldo <[email protected]>

Updated example repo to call out crypto example

17249e2

Signed-off-by: Whit Waldo <[email protected]>

Merge branch 'release-1.16' into crypto-v2

1fe0b1f

Signed-off-by: Whit Waldo <[email protected]>

Added cryptography to E2E test

b97692a

Signed-off-by: Whit Waldo <[email protected]>

Merge remote-tracking branch 'origin/crypto-v2' into crypto-v2

718d0cf

Removed non-existent project from solution

7e91bc9

Signed-off-by: Whit Waldo <[email protected]>

Fixing build issues regarding solution composition

542e328

Signed-off-by: Whit Waldo <[email protected]>

Removed unused dependency package

bfa77f2

Signed-off-by: Whit Waldo <[email protected]>

Removed specific build information from test project as it's provided…

828b40a

… for in a separate file Signed-off-by: Whit Waldo <[email protected]>

Fixed project imports, namespace and nullability annotation

b8c6adf

Signed-off-by: Whit Waldo <[email protected]>

watfordsuzy reviewed May 1, 2025

View reviewed changes

daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/_index.md Outdated Show resolved Hide resolved

watfordsuzy reviewed May 1, 2025

View reviewed changes

daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/dotnet-cryptography-usage.md Outdated Show resolved Hide resolved

watfordsuzy reviewed May 1, 2025

View reviewed changes

examples/Cryptography/Examples/EncryptDecryptLargeFileExample.cs Outdated Show resolved Hide resolved

WhitWaldo and others added 4 commits May 1, 2025 14:32

Temporarily disabled Crypto E2E tests

da4080e

Signed-off-by: Whit Waldo <[email protected]>

Update daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/_index.md

12b04fa

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Update daprdocs/content/en/dotnet-sdk-docs/dotnet-cryptography/dotnet…

9edc361

…-cryptography-usage.md Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Update examples/Cryptography/Examples/EncryptDecryptLargeFileExample.cs

7b4f16e

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

watfordsuzy reviewed May 1, 2025

View reviewed changes

WhitWaldo and others added 10 commits May 1, 2025 16:45

Removed bad references in test

e7b21ed

Signed-off-by: Whit Waldo <[email protected]>

Merge remote-tracking branch 'origin/crypto-v2' into crypto-v2

74173d6

Update src/Dapr.Cryptography/Encryption/Models/DecryptionOptions.cs

d22ff34

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Removed unnecessary block from docs

bdf12c1

Signed-off-by: Whit Waldo <[email protected]>

Added keys path to app output

ff77707

Signed-off-by: Whit Waldo <[email protected]>

Added exception handling to the encryption and decryption clients.

3b41723

Signed-off-by: Whit Waldo <[email protected]>

Added try/finally to unsubscribe the event handler for exception thro…

01ea04f

…wing Signed-off-by: Whit Waldo <[email protected]>

Reenabling E2E crypto tests

66bd321

Signed-off-by: Whit Waldo <[email protected]>

Upping the timeout

b082a14

Signed-off-by: Whit Waldo <[email protected]>

Removing crypto integration tests for now

8a1eac8

Signed-off-by: Whit Waldo <[email protected]>

watfordsuzy reviewed May 2, 2025

View reviewed changes

src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs Outdated Show resolved Hide resolved

src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs Outdated Show resolved Hide resolved

src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs Outdated Show resolved Hide resolved

WhitWaldo and others added 6 commits May 2, 2025 13:22

Update src/Dapr.Cryptography/Encryption/DecryptionStreamProcessor.cs

931962e

Co-authored-by: Christopher Watford <[email protected]> Signed-off-by: Whit Waldo <[email protected]>

Added exception handler for cancellation token timeouts, per review s…

8d06784

…uggestion. Signed-off-by: Whit Waldo <[email protected]>

Adopted review suggestion to handle request/channel writes in a final…

94a3d91

…ly block in case of exception Signed-off-by: Whit Waldo <[email protected]>

Minor cleanup to switch to a using declaration instead for clarity as…

3fcfa62

… the scope was no longer effective. Signed-off-by: Whit Waldo <[email protected]>

Sprinkle in some last-minute comments

7086d03

Signed-off-by: Whit Waldo <[email protected]>

Final touch-ups: remove unused usings, fixed an erroneous tab.

f33590b

Signed-off-by: Whit Waldo <[email protected]>

WhitWaldo merged commit 49992f4 into dapr:release-1.16 May 2, 2025
1 check passed

WhitWaldo deleted the crypto-v2 branch May 2, 2025 20:45

WhitWaldo mentioned this pull request May 2, 2025

Fix for large file cryptography support #1528

Merged

3 tasks

Add Dapr.Cryptography package + fix for large files #1527

Add Dapr.Cryptography package + fix for large files #1527

Uh oh!

Conversation

WhitWaldo commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issue reference

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

watfordsuzy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

watfordsuzy May 1, 2025

Choose a reason for hiding this comment

Uh oh!

WhitWaldo May 1, 2025

Choose a reason for hiding this comment

Uh oh!

watfordsuzy May 1, 2025

Choose a reason for hiding this comment

Uh oh!

WhitWaldo May 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

WhitWaldo commented May 1, 2025 •

edited

Loading