Support serializing traits into specification extensions in OpenAPI (#…

…1609)

## `smithy.openapi#specificationExtension`

Adds a meta trait `smithy.openapi#specificationExtension` that can be used to annotate traits to indicate they should be serialized into specification extension (`x-*`) properties when converting to OpenAPI. This is supported on shapes, operations, and services. By default the extension will be named by the shape ID replacing `#` & `.` with `-` prefixed with `x-`, otherwise the extension can be specified using the `as` property.

A new package `smithy-openapi-traits` is introduced to contain the `smithy.openapi#specificationExtension` trait.

## `JsonSchemaMapper` and `JsonSchemaShapeVisitor`

BREAKING CHANGE: Technically, `JsonSchemaMapper` has a breaking change from a functional interface to a normal interface, but we are anticipating customers are not using `JsonSchemaMapper` as a functional interface since it was not annotated with `@FunctionalInterface`.

`JsonSchemaMapper` is updated to use `updateSchema(JsonSchemaMapperContext, Schema.Builder)` in `JsonSchemaShapeVisitor`, which will call the existing `updateSchema(Shape, Schema.Builder, JsonSchemaConfig)` by default when not implemented for backwards compatibility.

## `smithy-openapi`

Support is added for `smithy.openapi#specificationExtension` by implementing `SpecificationExtensionsMapper` for operations and services and updating `OpenApiJsonSchemaMapper` for shapes.

---------

Co-authored-by: Steven Yuan <[email protected]>

docs/source-2.0/guides/converting-to-openapi.rst

@@ -1151,6 +1151,157 @@ The following is an example OpenAPI model for the above Smithy example value.
             }
         }
 
+-------------------------
+OpenAPI conversion traits
+-------------------------
+
+The ``software.amazon.smithy:smithy-openapi-traits`` package defines traits used to augment the conversion
+of a Smithy model into an OpenAPI specification.
+
+The following example shows how to add it to your Gradle build alongside the ``smithy-openapi`` plugin:
+
+.. code-block:: kotlin
+    :caption: build.gradle.kts
+
+    plugins {
+        java
+        id("software.amazon.smithy").version("0.6.0")
+    }
+
+    buildscript {
+        dependencies {
+            classpath("software.amazon.smithy:smithy-openapi:__smithy_version__")
+            classpath("software.amazon.smithy:smithy-openapi-traits:__smithy_version__")
+        }
+    }
+
+    dependencies {
+        implementation("software.amazon.smithy:smithy-openapi-traits:__smithy_version__")
+    }
+
+Refer to `Converting to OpenAPI with smithy-build`_ for more detailed information about using the plugin and Gradle.
+
+.. smithy-trait:: smithy.openapi#specificationExtension
+.. _specification-extension-trait:
+
+``specificationExtension`` trait
+================================
+
+Summary
+    Indicates a trait shape should be converted into an `OpenAPI specification extension`_.
+    Any custom trait that has been annotated with this trait will be serialized into the OpenAPI specification using
+    its :ref:`Smithy JSON AST representation <json-ast>`.
+Trait selector
+    ``[trait|trait]``
+Value type
+    ``structure``
+
+The ``specificationExtension`` trait is a structure that supports the following members:
+
+.. list-table::
+    :header-rows: 1
+    :widths: 10 25 65
+
+    * - Property
+      - Type
+      - Description
+    * - as
+      - ``string``
+      - Explicitly name the specification extension.
+        If set, it must begin with ``"x-"``.
+        Otherwise, it defaults to the target trait's shape ID normalized with hyphens and prepended with ``"x-"``.
+
+The following example defines a specification extension representing a custom metadata structure using the ``specificationExtension`` trait:
+
+.. code-block:: smithy
+
+    $version: "2"
+    namespace smithy.example
+
+    use smithy.openapi#specificationExtension
+
+    @trait
+    @specificationExtension(as: "x-meta")
+    structure metadata {
+        owner: String
+    }
+
+    @output
+    @metadata(owner: "greetings-team-b")
+    structure GreetResponse {
+        greeting: String
+    }
+
+    @readonly
+    @http(method: "GET", uri: "/greet")
+    @metadata(owner: "greetings-team-a")
+    operation Greet {
+        output: GreetResponse
+    }
+
+This results in an ``x-meta`` property being added to the respective objects in the OpenAPI output:
+
+.. code-block:: json
+
+    {
+        "...": "...",
+        "paths": {
+            "/greet": {
+                "get": {
+                    "operationId": "Greet",
+                    "responses": {
+                        "200": {
+                            "description": "Greet 200 response",
+                            "content": {
+                                "application/json": {
+                                    "schema": {
+                                        "$ref": "#/components/schemas/GreetResponseContent"
+                                    }
+                                }
+                            }
+                        }
+                    },
+                    "x-meta": {
+                        "owner": "greetings-team-a"
+                    }
+                }
+            }
+        },
+        "components": {
+            "schemas": {
+                "GreetResponseContent": {
+                    "type": "object",
+                    "properties": {
+                        "greeting": {
+                            "type": "string"
+                        }
+                    },
+                    "x-meta": {
+                        "owner": "greetings-team-b"
+                    }
+                }
+            }
+        }
+    }
+
+.. rubric:: Supported trait locations:
+
+Only a subset of OpenAPI locations are supported in the conversion:
+
+.. list-table::
+    :header-rows: 1
+    :widths: 50 50
+
+    * - Smithy Location
+      - OpenAPI Location
+    * - Service shape
+      - `Root OpenAPI schema <https://spec.openapis.org/oas/v3.1.0#openapi-object>`_
+    * - Operation shape
+      - `Operation object <https://spec.openapis.org/oas/v3.1.0#operation-object>`_
+    * - Simple & Aggregate shapes
+      - `Schema object <https://spec.openapis.org/oas/v3.1.0#schema-object>`_
+
+Unsupported use cases can likely be covered by the :ref:`jsonAdd <generate-openapi-setting-jsonAdd>` feature of the ``smithy-openapi`` plugin.
 
 -----------------------------
 Amazon API Gateway extensions
@@ -1754,3 +1905,4 @@ The conversion process is highly extensible through
 .. _x-amazon-apigateway-authorizer: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-authorizer.html
 .. _Lambda authorizers: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-authorizer.html
 .. _API Gateway's API key usage plans: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-usage-plans.html
+.. _OpenAPI specification extension: https://spec.openapis.org/oas/v3.1.0#specification-extensions

settings.gradle

@@ -21,6 +21,7 @@ include ":smithy-linters"
 include ":smithy-mqtt-traits"
 include ":smithy-jsonschema"
 include ":smithy-openapi"
+include ":smithy-openapi-traits"
 include ":smithy-utils"
 include ":smithy-protocol-test-traits"
 include ':smithy-jmespath'

smithy-jsonschema/src/main/java/software/amazon/smithy/jsonschema/JsonSchemaMapper.java

@@ -19,6 +19,11 @@
 
 /**
  * Updates a schema builder before converting a shape to a schema.
+ *
+ * {@link JsonSchemaMapper#updateSchema(JsonSchemaMapperContext, Schema.Builder)} is the entry point during JSON Schema
+ * conversion, and is the recommended method to implement. If this method is implemented,
+ * {@link JsonSchemaMapper#updateSchema(Shape, Schema.Builder, JsonSchemaConfig)} will NOT be called unless written in
+ * the implementation.
  */
 public interface JsonSchemaMapper {
     /**
@@ -36,12 +41,33 @@ default byte getOrder() {
     }
 
     /**
-     * Updates a schema builder.
+     * Updates a schema builder using information in {@link JsonSchemaMapperContext}.
+     *
+     * If not implemented, will default to
+     * {@link JsonSchemaMapper#updateSchema(Shape, Schema.Builder, JsonSchemaConfig)} for backwards-compatibility.
+     *
+     * @param context Context with information needed to update the schema.
+     * @param schemaBuilder Schema builder to update.
+     * @return Returns an updated schema builder.
+     */
+    default Schema.Builder updateSchema(JsonSchemaMapperContext context, Schema.Builder schemaBuilder) {
+        return updateSchema(context.getShape(), schemaBuilder, context.getConfig());
+    }
+
+    /**
+     * Updates a schema builder, and is not recommended. Use
+     * {@link JsonSchemaMapper#updateSchema(JsonSchemaMapperContext, Schema.Builder)} instead.
+     *
+     * If not implemented, this method will default to a no-op.
+     *
+     * This method is not deprecated for backwards-compatibility.
      *
      * @param shape Shape used for the conversion.
      * @param schemaBuilder Schema builder to update.
      * @param config JSON Schema config.
      * @return Returns an updated schema builder.
      */
-    Schema.Builder updateSchema(Shape shape, Schema.Builder schemaBuilder, JsonSchemaConfig config);
+    default Schema.Builder updateSchema(Shape shape, Schema.Builder schemaBuilder, JsonSchemaConfig config) {
+        return schemaBuilder;
+    }
 }

smithy-jsonschema/src/main/java/software/amazon/smithy/jsonschema/JsonSchemaMapperContext.java

@@ -0,0 +1,55 @@
+/*
+ * Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+package software.amazon.smithy.jsonschema;
+
+import software.amazon.smithy.model.Model;
+import software.amazon.smithy.model.shapes.Shape;
+
+/**
+ * Context for a JSON schema mapping.
+ */
+public class JsonSchemaMapperContext {
+    private final Model model;
+    private final Shape shape;
+    private final JsonSchemaConfig config;
+
+    JsonSchemaMapperContext(
+            Model model,
+            Shape shape,
+            JsonSchemaConfig config
+    ) {
+        this.model = model;
+        this.shape = shape;
+        this.config = config;
+    }
+
+    /**
+     * Gets the Smithy model being converted.
+     *
+     * @return Returns the Smithy model.
+     */
+    public Model getModel() {
+        return model;
+    }
+
+    /**
+     * Gets the Smithy shape being mapped.
+     *
+     * @return Returns the Smithy shape.
+     */
+    public Shape getShape() {
+        return shape;
+    }
+
+    /**
+     * Gets the JSON schema configuration object.
+     *
+     * @return Returns the JSON schema config object.
+     */
+    public JsonSchemaConfig getConfig() {
+        return config;
+    }
+}

smithy-jsonschema/src/main/java/software/amazon/smithy/jsonschema/JsonSchemaShapeVisitor.java

@@ -344,8 +344,9 @@ private Schema.Builder updateBuilder(Shape shape, Schema.Builder builder) {
      * @return Returns the built schema.
      */
     private Schema buildSchema(Shape shape, Schema.Builder builder) {
+        JsonSchemaMapperContext context = new JsonSchemaMapperContext(model, shape, converter.getConfig());
         for (JsonSchemaMapper mapper : mappers) {
-            mapper.updateSchema(shape, builder, converter.getConfig());
+            builder = mapper.updateSchema(context, builder);
         }
 
         return builder.build();

smithy-jsonschema/src/test/java/software/amazon/smithy/jsonschema/JsonSchemaConverterTest.java

@@ -211,7 +211,30 @@ public void canUseCustomPropertyNamingStrategy() {
     public void canAddCustomSchemaMapper() {
         Shape struct = StructureShape.builder().id("smithy.example#Foo").build();
         Model model = Model.builder().addShape(struct).build();
-        JsonSchemaMapper mapper = (shape, builder, conf) -> builder.putExtension("Hi", Node.from("There"));
+        class CustomMapper implements JsonSchemaMapper {
+            @Override
+            public Schema.Builder updateSchema(Shape shape, Schema.Builder builder, JsonSchemaConfig conf) {
+                return builder.putExtension("Hi", Node.from("There"));
+            }
+        }
+        JsonSchemaMapper mapper = new CustomMapper();
+        SchemaDocument doc = JsonSchemaConverter.builder().addMapper(mapper).model(model).build().convert();
+
+        assertTrue(doc.getDefinition("#/definitions/Foo").isPresent());
+        assertTrue(doc.getDefinition("#/definitions/Foo").get().getExtension("Hi").isPresent());
+    }
+
+    @Test
+    public void canAddCustomSchemaMapperContextMethod() {
+        Shape struct = StructureShape.builder().id("smithy.example#Foo").build();
+        Model model = Model.builder().addShape(struct).build();
+        class CustomMapper implements JsonSchemaMapper {
+            @Override
+            public Schema.Builder updateSchema(JsonSchemaMapperContext context, Schema.Builder builder) {
+                return builder.putExtension("Hi", Node.from("There"));
+            }
+        }
+        JsonSchemaMapper mapper = new CustomMapper();
         SchemaDocument doc = JsonSchemaConverter.builder().addMapper(mapper).model(model).build().convert();
 
         assertTrue(doc.getDefinition("#/definitions/Foo").isPresent());

smithy-openapi-traits/build.gradle

@@ -0,0 +1,15 @@
+/*
+ * Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ */
+
+description = "This module provides Smithy traits that are used in converting a Smithy model to OpenAPI."
+
+ext {
+    displayName = "Smithy :: OpenAPI Traits"
+    moduleName = "software.amazon.smithy.openapi.traits"
+}
+
+dependencies {
+    api project(":smithy-model")
+}