Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support server event streams #1479

Merged
merged 31 commits into from
Jul 25, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
7ec3c9c
Support server event streams
82marbag Jun 13, 2022
49e4cc5
fix function signature
82marbag Jul 14, 2022
cd3fa0d
revert tests for python
82marbag Jul 14, 2022
9c60845
update event stream marshalling tests
82marbag Jul 14, 2022
8f7e03f
remove todo on pokemon server
82marbag Jul 14, 2022
54ffd3f
document sign_empty None
82marbag Jul 15, 2022
a96ce2c
add documentation and rfc
82marbag Jul 15, 2022
dd1856b
update tests
82marbag Jul 18, 2022
dcc0dac
update tests
82marbag Jul 18, 2022
16f6fd9
address comments
82marbag Jul 18, 2022
09e9f31
update tests
82marbag Jul 18, 2022
2c700a5
render errors once
82marbag Jul 18, 2022
0ff3c5d
fix PythonCodegenServerPlugin
82marbag Jul 18, 2022
f1c932f
update tests
82marbag Jul 18, 2022
284e93d
update rfc
82marbag Jul 18, 2022
e1ff6e6
address comments
82marbag Jul 19, 2022
37f38e2
Merge branch 'main' into eventstreams
82marbag Jul 19, 2022
74fc001
address comments
82marbag Jul 19, 2022
011e02a
address comments
Jul 19, 2022
193ae52
render from errors once
82marbag Jul 19, 2022
6f77fb5
remove unused import in test
82marbag Jul 19, 2022
ecc60f4
refactor errors, generate on demand
82marbag Jul 21, 2022
6173c6e
Merge branch 'main' into eventstreams
82marbag Jul 21, 2022
5a10f87
Merge branch 'main' into eventstreams
82marbag Jul 21, 2022
0fce671
update CHANGELOG
82marbag Jul 21, 2022
4f51dd4
update TopLevelErrorGenerator
82marbag Jul 21, 2022
b446403
update CHANGELOG
82marbag Jul 22, 2022
5779a0d
Sort inline dependencies alphabetically by key
jdisanti Jul 21, 2022
f7fe78d
Revert "Sort inline dependencies alphabetically by key"
Jul 25, 2022
8f6ba8c
Merge branch 'main' into eventstreams
82marbag Jul 25, 2022
f389a45
rename rfc file
82marbag Jul 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 133 additions & 1 deletion CHANGELOG.next.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,136 @@
# message = "Fix typos in module documentation for generated crates"
# references = ["smithy-rs#920"]
# meta = { "breaking" = false, "tada" = false, "bug" = false, "sdk" = "client | server | all"}
# author = "rcoh"
# author = "rcoh"

[[smithy-rs]]
message = "Rename EventStreamInput to EventStreamSender"
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"

[[aws-sdk-rust]]
message = "Rename EventStreamInput to EventStreamSender"
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"

[[aws-sdk-rust]]
message = """
The type of streaming unions that contain errors is generated without those errors.
Errors in a streaming union `Union` are generated as members of the type `UnionError`.
Taking Transcribe as an example, the `AudioStream` streaming union generates, in the client, both the `AudioStream` type:
```rust
pub enum AudioStream {
AudioEvent(crate::model::AudioEvent),
Unknown,
}
```
and its error type,
```rust
pub struct AudioStreamError {
/// Kind of error that occurred.
pub kind: AudioStreamErrorKind,
/// Additional metadata about the error, including error code, message, and request ID.
pub(crate) meta: aws_smithy_types::Error,
}
```
`AudioStreamErrorKind` contains all error variants for the union.
Before, the generated code looked as:
```rust
pub enum AudioStream {
AudioEvent(crate::model::AudioEvent),
... all error variants,
Unknown,
}
```
"""
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"

[[aws-sdk-rust]]
message = """
`aws_smithy_http::event_stream::EventStreamSender` and `aws_smithy_http::event_stream::Receiver` are now generic over `<T, E>`,
where `T` is a streaming union and `E` the union's errors.
This means that event stream errors are now sent as `Err` of the union's error type.
With this example model:
```smithy
@streaming union Event {
throttlingError: ThrottlingError
}
@error("client") structure ThrottlingError {}
```
Before:
```rust
stream! { yield Ok(Event::ThrottlingError ...) }
```
After:
```rust
stream! { yield Err(EventError::ThrottlingError ...) }
```
An example from the SDK is in [transcribe streaming](https://github.com/awslabs/smithy-rs/blob/4f51dd450ea3234a7faf481c6025597f22f03805/aws/sdk/integration-tests/transcribestreaming/tests/test.rs#L80).
"""
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"

[[smithy-rs]]
82marbag marked this conversation as resolved.
Show resolved Hide resolved
message = """
The type of streaming unions that contain errors is generated without those errors.
Errors in a streaming union `Union` are generated as members of the type `UnionError`.
Taking Transcribe as an example, the `AudioStream` streaming union generates, in the client, both the `AudioStream` type:
```rust
pub enum AudioStream {
AudioEvent(crate::model::AudioEvent),
Unknown,
}
```
and its error type,
```rust
pub struct AudioStreamError {
/// Kind of error that occurred.
pub kind: AudioStreamErrorKind,
/// Additional metadata about the error, including error code, message, and request ID.
pub(crate) meta: aws_smithy_types::Error,
}
```
`AudioStreamErrorKind` contains all error variants for the union.
Before, the generated code looked as:
```rust
pub enum AudioStream {
AudioEvent(crate::model::AudioEvent),
... all error variants,
Unknown,
}
```
"""
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"

[[smithy-rs]]
message = """
`aws_smithy_http::event_stream::EventStreamSender` and `aws_smithy_http::event_stream::Receiver` are now generic over `<T, E>`,
where `T` is a streaming union and `E` the union's errors.
This means that event stream errors are now sent as `Err` of the union's error type.
With this example model:
```smithy
@streaming union Event {
throttlingError: ThrottlingError
}
@error("client") structure ThrottlingError {}
```
Before:
```rust
stream! { yield Ok(Event::ThrottlingError ...) }
```
After:
```rust
stream! { yield Err(EventError::ThrottlingError ...) }
```
An example from the SDK is in [transcribe streaming](https://github.com/awslabs/smithy-rs/blob/4f51dd450ea3234a7faf481c6025597f22f03805/aws/sdk/integration-tests/transcribestreaming/tests/test.rs#L80).
"""
references = ["smithy-rs#1157"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "82marbag"
4 changes: 2 additions & 2 deletions aws/rust-runtime/aws-sig-auth/src/event_stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ impl SignMessage for SigV4Signer {
Ok(signed_message)
}

fn sign_empty(&mut self) -> Result<Message, SignMessageError> {
fn sign_empty(&mut self) -> Option<Result<Message, SignMessageError>> {
let properties = self.properties.acquire();
if self.last_signature.is_none() {
// The Signature property should exist in the property bag for all Event Stream requests.
Expand All @@ -83,7 +83,7 @@ impl SignMessage for SigV4Signer {
sign_empty_message(self.last_signature.as_ref().unwrap(), &params).into_parts()
};
self.last_signature = Some(signature);
Ok(signed_message)
Some(Ok(signed_message))
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ class AwsInputPresignedMethod(
}

private fun RustWriter.writeInputPresignedMethod(section: OperationSection.InputImpl) {
val operationError = operationShape.errorSymbol(symbolProvider)
val operationError = operationShape.errorSymbol(coreCodegenContext.model, symbolProvider, coreCodegenContext.target)
val presignableOp = PRESIGNABLE_OPERATIONS.getValue(operationShape.id)

val makeOperationOp = if (presignableOp.hasModelTransforms()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import software.amazon.smithy.model.shapes.OperationShape
import software.amazon.smithy.model.shapes.ServiceShape
import software.amazon.smithy.model.shapes.ShapeId
import software.amazon.smithy.model.traits.OptionalAuthTrait
import software.amazon.smithy.rust.codegen.rustlang.CargoDependency
import software.amazon.smithy.rust.codegen.rustlang.Writable
import software.amazon.smithy.rust.codegen.rustlang.asType
import software.amazon.smithy.rust.codegen.rustlang.rust
Expand All @@ -27,7 +26,7 @@ import software.amazon.smithy.rust.codegen.smithy.customize.OperationCustomizati
import software.amazon.smithy.rust.codegen.smithy.customize.OperationSection
import software.amazon.smithy.rust.codegen.smithy.customize.RustCodegenDecorator
import software.amazon.smithy.rust.codegen.smithy.generators.config.ConfigCustomization
import software.amazon.smithy.rust.codegen.smithy.generators.config.ServiceConfig
import software.amazon.smithy.rust.codegen.smithy.generators.config.EventStreamSigningConfig
import software.amazon.smithy.rust.codegen.smithy.letIf
import software.amazon.smithy.rust.codegen.util.dq
import software.amazon.smithy.rust.codegen.util.expectTrait
Expand Down Expand Up @@ -82,51 +81,45 @@ class SigV4SigningConfig(
runtimeConfig: RuntimeConfig,
private val serviceHasEventStream: Boolean,
private val sigV4Trait: SigV4Trait
) : ConfigCustomization() {
) : EventStreamSigningConfig(runtimeConfig) {
private val codegenScope = arrayOf(
"SigV4Signer" to RuntimeType(
"SigV4Signer",
runtimeConfig.awsRuntimeDependency("aws-sig-auth", setOf("sign-eventstream")),
"aws_sig_auth::event_stream"
),
"SharedPropertyBag" to RuntimeType(
"SharedPropertyBag",
CargoDependency.SmithyHttp(runtimeConfig),
"aws_smithy_http::property_bag"
)
)

override fun section(section: ServiceConfig): Writable {
return when (section) {
is ServiceConfig.ConfigImpl -> writable {
override fun configImplSection(): Writable {
return writable {
rustTemplate(
"""
/// The signature version 4 service signing name to use in the credential scope when signing requests.
///
/// The signing service may be overridden by the `Endpoint`, or by specifying a custom
/// [`SigningService`](aws_types::SigningService) during operation construction
pub fn signing_service(&self) -> &'static str {
${sigV4Trait.name.dq()}
}
""",
*codegenScope
)
if (serviceHasEventStream) {
rustTemplate(
"""
/// The signature version 4 service signing name to use in the credential scope when signing requests.
///
/// The signing service may be overridden by the `Endpoint`, or by specifying a custom
/// [`SigningService`](aws_types::SigningService) during operation construction
pub fn signing_service(&self) -> &'static str {
${sigV4Trait.name.dq()}
}
""",
*codegenScope
)
if (serviceHasEventStream) {
rustTemplate(
"""
/// Creates a new Event Stream `SignMessage` implementor.
pub fn new_event_stream_signer(
&self,
properties: #{SharedPropertyBag}
) -> #{SigV4Signer} {
#{SigV4Signer}::new(properties)
"#{signerFn:W}",
"signerFn" to
renderEventStreamSignerFn { propertiesName ->
writable {
rustTemplate(
"""
#{SigV4Signer}::new($propertiesName)
""",
*codegenScope
)
}
}
""",
*codegenScope
)
}
)
}
else -> emptySection
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"test"
],
"content-type": [
"application/json"
"application/vnd.amazon.eventstream"
]
},
"method": "POST"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"AWS4-HMAC-SHA256 Credential=test/test/us-west-2/transcribe/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-user-agent;x-amzn-transcribe-language-code;x-amzn-transcribe-media-encoding;x-amzn-transcribe-sample-rate, Signature=test"
],
"content-type": [
"application/json"
"application/vnd.amazon.eventstream"
],
"x-amzn-transcribe-language-code": [
"en-GB"
Expand Down
9 changes: 4 additions & 5 deletions aws/sdk/integration-tests/transcribestreaming/tests/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

use async_stream::stream;
use aws_sdk_transcribestreaming::error::{
StartStreamTranscriptionError, StartStreamTranscriptionErrorKind,
AudioStreamError, TranscriptResultStreamError, TranscriptResultStreamErrorKind,
};
use aws_sdk_transcribestreaming::model::{
AudioEvent, AudioStream, LanguageCode, MediaEncoding, TranscriptResultStream,
Expand All @@ -15,7 +15,6 @@ use aws_sdk_transcribestreaming::types::{Blob, SdkError};
use aws_sdk_transcribestreaming::{Client, Config, Credentials, Region};
use aws_smithy_client::dvr::{Event, ReplayingConnection};
use aws_smithy_eventstream::frame::{DecodedFrame, HeaderValue, Message, MessageFrameDecoder};
use aws_smithy_http::event_stream::BoxError;
use bytes::BufMut;
use futures_core::Stream;
use std::collections::{BTreeMap, BTreeSet};
Expand Down Expand Up @@ -78,8 +77,8 @@ async fn test_error() {
match output.transcript_result_stream.recv().await {
Err(SdkError::ServiceError {
err:
StartStreamTranscriptionError {
kind: StartStreamTranscriptionErrorKind::BadRequestException(err),
TranscriptResultStreamError {
kind: TranscriptResultStreamErrorKind::BadRequestException(err),
..
},
..
Expand All @@ -102,7 +101,7 @@ async fn test_error() {
async fn start_request(
region: &'static str,
events_json: &str,
input_stream: impl Stream<Item = Result<AudioStream, BoxError>> + Send + Sync + 'static,
input_stream: impl Stream<Item = Result<AudioStream, AudioStreamError>> + Send + Sync + 'static,
) -> (ReplayingConnection, StartStreamTranscriptionOutput) {
let events: Vec<Event> = serde_json::from_str(events_json).unwrap();
let replayer = ReplayingConnection::new(events);
Expand Down
Loading