Add S3 Express identity cache #3390
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds supporting types for S3 Express. They are provided via the S3 customization and defining Rust types live in `aws-inlineable`.
This commit updates parts of the orchestrator so that when an S3 Express bucket name is passed, control flow will be directed to placeholder types added in the previous commit.
…/customize/s3/S3ExpressDecorator.kt Co-authored-by: John DiSanti <[email protected]>
This commit addresses #3386 (comment)
This commit addresses #3386 (comment)
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
rcoh
reviewed
Feb 6, 2024
rcoh
reviewed
Feb 6, 2024
rcoh
reviewed
Feb 6, 2024
rcoh
reviewed
Feb 6, 2024
rcoh
reviewed
Feb 6, 2024
...dk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/customize/s3/S3ExpressDecorator.kt
Outdated
Show resolved
Hide resolved
rcoh
reviewed
Feb 6, 2024
...dk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/customize/s3/S3ExpressDecorator.kt
Outdated
Show resolved
Hide resolved
rcoh
reviewed
Feb 6, 2024
| dbg!(result).expect("success"); | ||
|
|
||
| http_client | ||
| .validate_body_and_headers(Some(&["x-amz-s3session-token"]), "application/xml") |
There was a problem hiding this comment.
if you want to validate this header, you can use a custom timesource to stop time. Then I think it will be deterministic?
There was a problem hiding this comment.
I believe a header x-amz-s3session-token is included for validate_body_and_headers, maybe you're referring to assert_requests_match?
rcoh
reviewed
Feb 6, 2024
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
This commit addresses #3390 (comment)
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
Base automatically changed from
s3express-allow-list-objects-v2-to-run
to
ysaito/s3express
February 17, 2024 02:33
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
jdisanti
reviewed
Feb 20, 2024
jdisanti
approved these changes
Feb 20, 2024
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
This commit addresses #3390 (comment)
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
1 task
github-merge-queue bot
pushed a commit
that referenced
this pull request
Mar 11, 2024
## Motivation and Context Allows the Rust SDK to use [S3 Express One Zone](https://aws.amazon.com/s3/storage-classes/express-one-zone/) ## Description The PR adds the said S3-specific functionality to the Rust SDK. The code changes have already been reviewed by previous sub PRs, but it's worth going through them again as a whole: - #3386 - #3388 - #3390 - #3432 - #3433 - #3459 - #3457 - #3462 In addition to the PRs above, commit eebe8af increases the canary lambda's memory size to 512MB from 128MB (also makes it configurable through a command line arg for `canary-runner`). By default, lambda's allowed memory size is 128MB but with the addition of `canary-wasm` in main, canary lambda's memory usage will be 152MB, causing the lambda to be killed by a signal during runtime. The commit addresses that issue. ## Testing - Unit tests in [aws/rust-runtime/aws-inlineable/src/s3_express.rs](https://github.com/smithy-lang/smithy-rs/blob/7f8c28b7038372927ec6196eff88384452f908dd/aws/rust-runtime/aws-inlineable/src/s3_express.rs) - Integration tests in [aws/sdk/integration-tests/s3/tests/express.rs](https://github.com/smithy-lang/smithy-rs/blob/7f8c28b7038372927ec6196eff88384452f908dd/aws/sdk/integration-tests/s3/tests/express.rs) - Canary in smithy-rs#3462 ## Checklist <!--- If a checkbox below is not applicable, then please DELETE it rather than leaving it unchecked --> - [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS SDK, generated SDK code, or SDK runtime crates ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._ --------- Co-authored-by: John DiSanti <[email protected]> Co-authored-by: AWS SDK Rust Bot <[email protected]> Co-authored-by: AWS SDK Rust Bot <[email protected]> Co-authored-by: Zelda Hessler <[email protected]> Co-authored-by: Russell Cohen <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Adds a default implementation for S3 Express identity cache.
Description
This PR adds the said cache for S3 Express. This cache is not configurable from outside and solely owned by the default S3 Express identity provider. It is implemented in terms of an LRU cache keyed on a string generated by
sha256hmac(random 64-byte key, access_key_id + secret_key) + bucket_name(note:access_key_idandsecret_keyare for a customer's credentials but not for a retrievedcreate_sessionAPI token).Cache values are of type
ExpiringCachethat contains a session token retrieved by S3'screate_sessionAPI. When a customer is trying to use a cached session token but if it has expired,ExpiringCachecalls the S3'screate_sessionAPI, stores in it a new session token, and returns it to the customer.Testing
Added unit tests for
S3IdentityCacheand a connection recording test forlist-objects-v2running against both express and regular buckets to exercise a use case where a customer is switching between those buckets.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.