Add server RPC v2 CBOR support #2544
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
6 tasks
david-perez
force-pushed
the
smithy-rpc-v2
branch
from
24d2614 to
b9016ae
Compare
|
Squashed all commits rebased on top of current |
This fixes two bugs: 1. `Content-Type` header checking was succeeding when no `Content-Type` header was present but one was expected. 2. When a shape was @httpPayload`-bound, `Content-Type` header checking occurred even when no payload was being sent. In this case it is not necessary to check the header, since there is no content. Code has been refactored and cleaned up. The crux of the logic is now easier to understand, and contained in `content_type_header_classifier`.
…test model; but switching to refactoring protocol test generation now
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
ysaito1001
approved these changes
Jul 11, 2024
There was a problem hiding this comment.
Reviewed all but codegen-server-test, codegen/server, and aws-smithy-http-server. Thank you for @jjant for this awesome work!
| * appear when serializing both. | ||
| * | ||
| * Strictly speaking, the spec says we should only add `__type` when serializing an operation error response, but | ||
| * there shouldn't™️ be any harm in always including it, which simplifies the code generator. |
There was a problem hiding this comment.
Suggested change
| * there shouldn't™️ be any harm in always including it, which simplifies the code generator. | |
| * there shouldn't be any harm in always including it, which simplifies the code generator. |
drganjoo
reviewed
Jul 16, 2024
...e/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/Instantiator.kt
Show resolved
Hide resolved
...-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/RpcV2Cbor.kt
Show resolved
Hide resolved
...otlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/ReturnSymbolToParse.kt
Outdated
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/JsonSerializerGenerator.kt
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
...otlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/CborParserGenerator.kt
Outdated
Show resolved
Hide resolved
...otlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/CborParserGenerator.kt
Outdated
Show resolved
Hide resolved
...otlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/CborParserGenerator.kt
Outdated
Show resolved
Hide resolved
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
drganjoo
approved these changes
Jul 16, 2024
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/JsonSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
| writable { | ||
| // We're only interested in _structure_ member shapes that can reach constrained shapes. | ||
| if ( | ||
| codegenContext.model.expectShape(section.shape.container) is StructureShape && |
There was a problem hiding this comment.
Yes, a Smithy shape is constrained, but somewhere in the codebase, we've already generated a Rust symbol that represents this fact. MaybeConstrained is a Rust type/symbol, not directly related to Smithy. My point is, we have 108 instances where we check if a Smithy shape is constrained. Instead, why don't we query the symbol provider or the symbol itself to determine if it supports the MaybeConstrained Rust type and if we need to issue a v.into() to convert to MaybeConstrained, rather than checking that a Shape's target can reach a constrained shape, and assuming we've generated a MaybeConstrained for that shape?
...ftware/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/CborSerializerGenerator.kt
Outdated
Show resolved
Hide resolved
|
A new generated diff is ready to view.
A new doc preview is ready to view. |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jul 31, 2024
## Motivation and Context Follow-up on #2544 to add client-side support for the protocol ## Description The client implementation mainly focuses on a sub-section [Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests) in the spec. To that end, this PR addresses `TODO` for the client to fill in the blanks and includes additional adjustments/refactoring to pass client protocol tests. ## Testing - Existing tests in CI - Upstream protocol test `rpcv2Cbor` - Our handwritten protocol test `rpcv2Cbor-extras.smithy` ---- _By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice._
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RPC v2 CBOR is a new protocol that
is being addedhas recently been added to the Smithyspecification.
(I'll add more details here as the patchset evolves)
Credit goes to @jjant for initial implementation of the router, which I built on top of from his
jjant/smithy-rpc-v2-explorationbranch.Tracking issue: #3573.
Caveats
TODOs are currently exhaustively sprinkled throughout the patch documenting what remains to be done. Most of these need to be addressed before this can be merged in; some can be punted on to not make this PR bigger.However, I'd like to call out the major caveats and blockers here. I'll keep updating this list as the patchset evolves.
smithy-rpc-v2branch. The following are prerrequisites for this PR to be merged; until they are done CI on this PR will fail:Protocol tests for the protocol do not currently exist in Smithy. Until those get written, this PR resorts to Rust unit tests and integration tests that useserdeto round-trip messages and compareserde's encoders and decoders with ours for correctness.smithy-protocol-testsdirectory in Smithy.serde_cborround-trip tests for defense in depth.[ ] Smithydocumentshapes are not supported. RPC v2's specification currently doesn't indicate how to implement them.Prerequisite PRs
This section lists prerequisite PRs and issues that would make the diff for this one lighter or easier to understand. It's preferable that these PRs be merged prior to this one; some are hard prerequisites. They mostly relate to parts of the codebase I've had to touch or
pilferinspect in this PR where I've made necessary changes, refactors and "drive-by improvements" that are mostly unrelated, although some directly unlock things I've needed in this patchset. It makes sense to pull them out to ease reviewability and make this patch more semantically self-contained.ignoreUnsupportedConstraintTraitsfrom integration test models #2516RuntimeErrorandRequestRejectionby protocol #2517aws_smithy_types::primitive::Parsein the server #2528ProtocolsstructuredDataParser,structuredDataSerializer#2536RestRequestSpecGeneratorintocodegen-server#2537Content-Typeheader in all server protocols #2531#[allow(unused_variables)]in JSON serializers when structure has no members #2538ignoreUnsupportedConstraintTraitshas no effect #2539RUSTFLAGSenvironment variable when invoking Cargo commands via Gradle tasks #3678Content-Typeheader checking in servers #3690@httpPayload#3752Testing
RPC v2 has still not been added to the Smithy specification. It is currently being worked on over in the
smithy-rpc-v2branch.This can only be tested locally following these steps:
1. Clone the Smithy repository and checkout thesmithy-rpc-v2branch.2. Inside your local checkout of smithy-rs pointing to this PR's branch, make sure you've added
mavenLocal()as a repository in thebuild.gradle.ktsfiles. Example.4. Inside your local checkout of Smithy's
smithy-rpc-v2branch:1. Set
VERSIONto the current Smithy version used in smithy-rs (1.28.1 as of writing, but check here).2. Run
./gradlew clean build pTML.6.1. In your local checkout of the smithy-rs'ssmithy-rpc-v2branch, run./gradlew codegen-server-test:build -P modules='rpcv2Cbor'.You can troubleshoot whether you have Smithy correctly set up locally by inspecting~/.m2/repository/software/amazon/smithy/smithy-protocols-traits.Checklist
CHANGELOG.next.tomlif I made changes to the smithy-rs codegen or runtime cratesBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.