Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor runtimeType usage in codegen #1933

Closed
wants to merge 25 commits into from
Closed
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
65e6ed9
move: all AWS decorators into a new decorators folder
Velfi Nov 1, 2022
8b668ad
move: all AWS-service-specific customizations into a new servicecusto…
Velfi Nov 1, 2022
813e388
fix: various small lints and compile warnings
Velfi Nov 1, 2022
f1e9e1f
refactor: unify how runtime types are instantiated
Velfi Nov 1, 2022
f652e30
fix: build.gradle.kts improper delegation issue
Velfi Nov 1, 2022
a47ab41
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 1, 2022
9979d76
fix: set default RuntimeConfig cratePrefix to "aws"
Velfi Nov 1, 2022
9a50129
add: another missing import
Velfi Nov 1, 2022
b8678ff
update: sha256 dep of crate-hasher tool
Velfi Nov 2, 2022
263d9c6
update: Cargo.lock of crate-hasher tool
Velfi Nov 2, 2022
52a55fd
update: make doc-only usage of awsConfig import it as a dev dependency
Velfi Nov 3, 2022
ced89c5
update: move SleepImplFromSdkConfig into ResiliencyDecorator
Velfi Nov 4, 2022
40759ad
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 4, 2022
7400922
remove: CHANGELOG.next.toml entries only relevant to smithy contribs
Velfi Nov 4, 2022
84acfd2
undo: decorator ordering change
Velfi Nov 8, 2022
840d13a
remove: code that didn't do anything
Velfi Nov 8, 2022
3e8a0a4
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 8, 2022
8d48f5c
fix: server module missing import
Velfi Nov 9, 2022
f651183
refactor: simplify RuntimeType
Velfi Nov 9, 2022
6daebef
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 9, 2022
ab09d99
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 9, 2022
1c95fb7
fix: outdated RuntimeType usage
Velfi Nov 9, 2022
7aae03d
rename: `RuntimeType.asType()` to `RuntimeType.toType()` in conventio…
Velfi Nov 10, 2022
2c0ed8b
update: derive attribute type sorting
Velfi Nov 10, 2022
558ddc8
Merge branch 'main' into refactor/unify-runtime-crate-usage-in-codegen
Velfi Nov 10, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion CHANGELOG.next.toml
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,3 @@ message = "Removed re-export of `aws_smithy_client::retry::Config` from the `mid
references = ["smithy-rs#1935"]
meta = { "breaking" = true, "tada" = false, "bug" = false }
author = "jdisanti"

3 changes: 1 addition & 2 deletions aws/sdk-adhoc-test/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ extra["moduleName"] = "software.amazon.smithy.kotlin.codegen.test"
tasks["jar"].enabled = false

plugins {
val smithyGradlePluginVersion: String by project
id("software.amazon.smithy").version(smithyGradlePluginVersion)
id("software.amazon.smithy")
}

val smithyVersion: String by project
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0
*/

package software.amazon.smithy.rustsdk

import software.amazon.smithy.rust.codegen.core.smithy.customize.NamedSectionGenerator
import software.amazon.smithy.rust.codegen.core.smithy.customize.Section

sealed class AwsSection(name: String) : Section(name) {
abstract val customizations: List<AwsCustomization>

/**
* Write custom code into the `impl From<&aws_types::sdk_config::SdkConfig> for Builder`.
* This `From` impl takes a reference; Don't forget to clone non-`Copy` types.
*/
data class FromSdkConfigForBuilder(override val customizations: List<AwsCustomization>) :
AwsSection("FromSdkConfig")
}

abstract class AwsCustomization : NamedSectionGenerator<AwsSection>()
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ package software.amazon.smithy.rustsdk
import software.amazon.smithy.codegen.core.CodegenException
import software.amazon.smithy.rust.codegen.core.rustlang.CargoDependency
import software.amazon.smithy.rust.codegen.core.rustlang.Visibility
import software.amazon.smithy.rust.codegen.core.rustlang.asType
import software.amazon.smithy.rust.codegen.core.smithy.RuntimeConfig
import software.amazon.smithy.rust.codegen.core.smithy.RuntimeCrateLocation
import software.amazon.smithy.rust.codegen.core.smithy.RuntimeType
Expand Down Expand Up @@ -50,20 +51,23 @@ object AwsRuntimeType {
fun RuntimeConfig.defaultMiddleware() = RuntimeType.forInlineDependency(
InlineAwsDependency.forRustFile(
"middleware", visibility = Visibility.PUBLIC,
CargoDependency.SmithyHttp(this),
CargoDependency.SmithyHttpTower(this),
CargoDependency.SmithyClient(this),
CargoDependency.smithyHttp(this),
CargoDependency.smithyHttpTower(this),
CargoDependency.smithyClient(this),
CargoDependency.Tower,
awsHttp(),
awsEndpoint(),
awsRuntimeCrate("aws-http"),
awsRuntimeCrate("aws-endpoint"),
),
).member("DefaultMiddleware")
}

fun RuntimeConfig.awsRuntimeDependency(name: String, features: Set<String> = setOf()): CargoDependency =
fun RuntimeConfig.awsRuntimeCrate(name: String, features: Set<String> = setOf()): CargoDependency =
CargoDependency(name, awsRoot().crateLocation(null), features = features)

fun RuntimeConfig.awsHttp(): CargoDependency = awsRuntimeDependency("aws-http")
fun RuntimeConfig.awsTypes(): CargoDependency = awsRuntimeDependency("aws-types")
fun RuntimeConfig.awsConfig(): CargoDependency = awsRuntimeDependency("aws-config")
fun RuntimeConfig.awsEndpoint() = awsRuntimeDependency("aws-endpoint")
fun RuntimeConfig.awsConfig() = awsRuntimeCrate("aws-config").asType()
fun RuntimeConfig.awsEndpoint() = awsRuntimeCrate("aws-endpoint").asType()
fun RuntimeConfig.awsHttp() = awsRuntimeCrate("aws-http").asType()
fun RuntimeConfig.awsSigAuth() = awsRuntimeCrate("aws-sig-auth").asType()
fun RuntimeConfig.awsSigAuthEventStream() = awsRuntimeCrate("aws-sig-auth", setOf("sign-eventstream")).asType()
fun RuntimeConfig.awsSigv4() = awsRuntimeCrate("aws-sigv4").asType()
fun RuntimeConfig.awsTypes() = awsRuntimeCrate("aws-types").asType()
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ class SdkSettings private constructor(private val awsSdk: ObjectNode?) {

/** Path to the `sdk-default-configuration.json` config file */
val defaultsConfigPath: Path? get() =
awsSdk?.getStringMember("defaultConfigPath")?.orNull()?.value.let { Paths.get(it) }
awsSdk?.getStringMember("defaultConfigPath")?.orNull()?.value?.let { Paths.get(it) }

/** Path to the `sdk-endpoints.json` configuration */
val endpointsConfigPath: Path? get() =
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0
*/

package software.amazon.smithy.rustsdk.decorators

import software.amazon.smithy.rust.codegen.client.smithy.ClientCodegenContext
import software.amazon.smithy.rust.codegen.client.smithy.customizations.DocsRsMetadataDecorator
import software.amazon.smithy.rust.codegen.client.smithy.customizations.DocsRsMetadataSettings
import software.amazon.smithy.rust.codegen.client.smithy.customize.CombinedCodegenDecorator
import software.amazon.smithy.rust.codegen.client.smithy.customize.RustCodegenDecorator
import software.amazon.smithy.rust.codegen.client.smithy.generators.protocol.ClientProtocolGenerator
import software.amazon.smithy.rust.codegen.core.rustlang.RustModule
import software.amazon.smithy.rust.codegen.core.rustlang.rust
import software.amazon.smithy.rust.codegen.core.rustlang.rustBlockTemplate
import software.amazon.smithy.rust.codegen.core.smithy.RuntimeConfig
import software.amazon.smithy.rust.codegen.core.smithy.RustCrate
import software.amazon.smithy.rust.codegen.core.smithy.customize.writeCustomizations
import software.amazon.smithy.rustsdk.AwsCustomization
import software.amazon.smithy.rustsdk.AwsSection
import software.amazon.smithy.rustsdk.awsTypes
import software.amazon.smithy.rustsdk.servicedecorators.apigateway.ApiGatewayDecorator
import software.amazon.smithy.rustsdk.servicedecorators.auth.DisabledAuthDecorator
import software.amazon.smithy.rustsdk.servicedecorators.ec2.Ec2Decorator
import software.amazon.smithy.rustsdk.servicedecorators.glacier.GlacierDecorator
import software.amazon.smithy.rustsdk.servicedecorators.route53.Route53Decorator
import software.amazon.smithy.rustsdk.servicedecorators.s3.S3Decorator
import software.amazon.smithy.rustsdk.servicedecorators.sts.STSDecorator

val DECORATORS = listOf(
Velfi marked this conversation as resolved.
Show resolved Hide resolved
// General AWS Decorators
AwsEndpointDecorator(),
AwsFluentClientDecorator(),
AwsPresigningDecorator(),
AwsReadmeDecorator(),
CrateLicenseDecorator(),
CredentialsProviderDecorator(),
HttpRequestChecksumDecorator(),
HttpResponseChecksumDecorator(),
IntegrationTestDecorator(),
RegionDecorator(),
ResiliencyDecorator(),
SdkConfigDecorator(),
ServiceConfigDecorator(),
SigV4SigningDecorator(),
UserAgentDecorator(),

// Service specific decorators
ApiGatewayDecorator(),
DisabledAuthDecorator(),
Ec2Decorator(),
GlacierDecorator(),
Route53Decorator(),
S3Decorator(),
STSDecorator(),

// Only build docs-rs for linux to reduce load on docs.rs
DocsRsMetadataDecorator(DocsRsMetadataSettings(targets = listOf("x86_64-unknown-linux-gnu"), allFeatures = true)),
)

interface AwsCodegenDecorator : RustCodegenDecorator<ClientProtocolGenerator, ClientCodegenContext> {
fun awsCustomizations(
codegenContext: ClientCodegenContext,
baseCustomizations: List<AwsCustomization>,
): List<AwsCustomization> {
return baseCustomizations
}
}

class AwsCombinedCodegenDecorator :
CombinedCodegenDecorator<ClientProtocolGenerator, ClientCodegenContext>(DECORATORS) {
override val order: Byte = -1

override fun extras(codegenContext: ClientCodegenContext, rustCrate: RustCrate) {
super.extras(codegenContext, rustCrate)

val awsCustomizations = orderedDecorators.filterIsInstance<AwsCodegenDecorator>()
.foldRight(listOf<AwsCustomization>()) { decorator, customizations ->
decorator.awsCustomizations(codegenContext, customizations)
}
generateImplFromRefSdkConfigForConfigBuilder(codegenContext.runtimeConfig, rustCrate, awsCustomizations)
}
}

fun generateImplFromRefSdkConfigForConfigBuilder(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be in SdkConfigDecorator?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think no because I feel that all the section writing code should be defined in one place. The combined decorator is also the place where we have access to the full list of decorators that modify an AwsSection.

Copy link
Collaborator

@jdisanti jdisanti Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Organizationally, this isn't going to work out well since people will think to look in SdkConfigDecorator for this code rather than in AwsCombinedCodegenDecorator. The full list of decorators can be passed into the customizations if it's required (and I think we do that in codegen-core/codegen-client).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Russell is adding the ability to do this as part of #1953, I can take the change from that or wait for the merge.

runtimeConfig: RuntimeConfig,
rustCrate: RustCrate,
customizations: List<AwsCustomization>,
) {
val codegenContext = arrayOf("SdkConfig" to runtimeConfig.awsTypes().member("sdk_config::SdkConfig"))

rustCrate.withModule(RustModule.Config) {
rustBlockTemplate("impl From<&#{SdkConfig}> for Builder", *codegenContext) {
rustBlockTemplate("fn from(input: &#{SdkConfig}) -> Self", *codegenContext) {
rust("let mut builder = Builder::default();")
writeCustomizations(customizations, AwsSection.FromSdkConfigForBuilder(customizations))
rust("builder")
}
}
}
}
Loading