-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feature: support flexible checksums #1561
Conversation
update: s3 model
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exciting to see all this come together! It looks like there's a server codegen compile error that's preventing CI from running.
use http_body::Body; | ||
|
||
// Ensure that a valid `checksum_algorithm` was passed, emitting an error if that's not the case | ||
if let Err(err) = aws_smithy_checksums::http::new_from_algorithm(&checksum_algorithm) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like we should create the algorithm once to avoid an unnecessary allocation since it returns a boxed value. It's being created up here for validation, and then again below for the actual checksum calculation.
Edit: I see the reason for this now in the code comment below. I think we should create an intermediate Algorithm
enum type that is easily converted to/from algorithm names and header names, and can easily create algorithm implementations. I prototyped this a bit and made it a macro so that there's no risk of any of the functions on the enum getting out of sync with each other:
macro_rules! associate_algorithms {
(pub enum $enum_name:ident { $($algorithm_ident:ident => { name: $algorithm_name:ident, header_name: $header_name:ident },)+ }) => {
#[derive(Copy, Clone)]
pub enum $enum_name {
$($algorithm_ident,)+
}
impl $enum_name {
pub fn to_header_name(&self) -> Result<HeaderName, Box<dyn std::error::Error>> {
match self {
$(Self::$algorithm_ident => Ok($header_name.clone()),)+
}
}
pub fn from_header_name(checksum_header_name: &HeaderName) -> Result<Self, Box<dyn std::error::Error>> {
$(if checksum_header_name == $header_name { return Ok(Self::$algorithm_ident) })+
Err(Box::new(Error::UnknownChecksumHeaderName(
checksum_header_name.to_owned(),
)))
}
pub fn to_algorithm_name(&self) -> &str {
match self {
$(Self::$algorithm_ident => $algorithm_name,)+
}
}
pub fn from_algorithm_name(algorithm_name: &str) -> Result<Self, Box<dyn std::error::Error>> {
$(if algorithm_name == $algorithm_name { return Ok(Self::$algorithm_ident) })+
Err(Box::new(Error::UnknownChecksumAlgorithm(
algorithm_name.to_owned(),
)))
}
pub fn to_impl(&self) -> Box<dyn HttpChecksum> {
match self {
$(Self::$algorithm_ident => Box::new($algorithm_ident::default()),)+
}
}
}
};
}
associate_algorithms!(
pub enum Algorithm {
Crc32 => { name: CRC_32_NAME, header_name: CRC_32_HEADER_NAME },
Crc32c => { name: CRC_32_C_NAME, header_name: CRC_32_C_HEADER_NAME },
Md5 => { name: MD5_NAME, header_name: MD5_HEADER_NAME },
Sha1 => { name: SHA_1_NAME, header_name: SHA_1_HEADER_NAME },
Sha256 => { name: SHA_256_NAME, header_name: SHA_256_HEADER_NAME },
}
);
Then we can verify an algorithm is supported with:
let algorithm = Algorithm::from_algorithm_name(&checksum_algorithm)
.map_err(|err| BuildError::Other(err))?;
This Algorithm
enum is Copy
, so we don't have to worry about it moving anywhere. Then inside the body checksumming, we easily get a fresh implementation when we need it:
let checksum = algorithm.to_impl();
No error checking required since we already error checked.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I implemented the enum idea, but not the macro. Let me know what you think.
aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/HttpResponseChecksumDecorator.kt
Outdated
Show resolved
Hide resolved
add: customizations arg to ServerHttpBoundProtocolTraitImplGenerator.generateTraitImpls
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
update: allow dead code in http_body_checksum.rs so we can use pub(crate)
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! just some small stylistic changes
aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/HttpResponseChecksumDecorator.kt
Outdated
Show resolved
Hide resolved
A new generated diff is ready to view.
A new doc preview is ready to view. |
A new generated diff is ready to view.
A new doc preview is ready to view. |
A new generated diff is ready to view.
A new doc preview is ready to view. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM from a server perspective. The diff is also empty, which is a good indicator!
A new generated diff is ready to view.
A new doc preview is ready to view. |
Motivation and Context
#1482
Description
This PR adds all the inlineables and codegen necessary to support flexible checksums. It also updates the S3 model to the latest version and tests the new checksum functionality against that model.
Testing
I have written many tests to ensure the code is correct.
Checklist
CHANGELOG.next.toml
if I made changes to the smithy-rs codegen or runtime cratesCHANGELOG.next.toml
if I made changes to the AWS SDK, generated SDK code, or SDK runtime cratesBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.