Add SXG filter extension #17215
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py. | ||
|
|
||
| load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package") | ||
|
|
||
| licenses(["notice"]) # Apache 2 | ||
|
|
||
| api_proto_package( | ||
| deps = [ | ||
| "//envoy/extensions/transport_sockets/tls/v3:pkg", | ||
| "@com_github_cncf_udpa//udpa/annotations:pkg", | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package envoy.extensions.filters.http.sxg.v3alpha; | ||
|
|
||
| import "envoy/extensions/transport_sockets/tls/v3/secret.proto"; | ||
|
|
||
| import "google/protobuf/duration.proto"; | ||
|
|
||
| import "udpa/annotations/status.proto"; | ||
| import "validate/validate.proto"; | ||
|
|
||
| option java_package = "io.envoyproxy.envoy.extensions.filters.http.sxg.v3alpha"; | ||
| option java_outer_classname = "SxgProto"; | ||
| option java_multiple_files = true; | ||
| option (udpa.annotations.file_status).work_in_progress = true; | ||
| option (udpa.annotations.file_status).package_version_status = ACTIVE; | ||
|
|
||
| // [#protodoc-title: Signed HTTP Exchange Filter] | ||
| // SXG :ref:`configuration overview <config_http_filters_sxg>`. | ||
| // [#extension: envoy.filters.http.sxg] | ||
|
|
||
| // [#next-free-field: 10] | ||
| message SXG { | ||
| // The SDS configuration for the public key data for the SSL certificate that will be used to sign the | ||
| // SXG response. | ||
| transport_sockets.tls.v3.SdsSecretConfig certificate = 1; | ||
|
|
||
| // The SDS configuration for the private key data for the SSL certificate that will be used to sign the | ||
| // SXG response. | ||
| transport_sockets.tls.v3.SdsSecretConfig private_key = 2; | ||
|
cpapazian marked this conversation as resolved.
Outdated
|
||
|
|
||
| // The duration for which the generated SXG package will be valid. Default is 604800s (7 days in seconds). | ||
| // Note that in order to account for clock skew, the timestamp will be backdated by a day. So, if duration | ||
| // is set to 7 days, that will be 7 days from 24 hours ago (6 days from now). Also note that while 6/7 days | ||
| // is appropriate for most content, if the downstream service is serving Javascript, or HTML with inline | ||
| // Javascript, 1 day (so, with backdated expiry, 2 days, or 172800 seconds) is more appropriate. | ||
| google.protobuf.Duration duration = 3; | ||
|
|
||
| // The SXG response payload is Merkle Integrity Content Encoding (MICE) encoded (specification is [here](https://datatracker.ietf.org/doc/html/draft-thomson-http-mice-03)) | ||
| // This value indicates the record size in the encoded payload. The default value is 4096. | ||
| uint64 mi_record_size = 4; | ||
|
|
||
| // The URI of certificate CBOR file published. Since it is required that the certificate CBOR file | ||
| // be served from the same domain as the SXG document, this should be a relative URI. | ||
| string cbor_url = 5 [(validate.rules).string = {min_len: 1 prefix: "/"}]; | ||
|
|
||
| // URL to retrieve validity data for signature, a CBOR map. See specification [here](https://tools.ietf.org/html/draft-yasskin-httpbis-origin-signed-exchanges-impl-00#section-3.6) | ||
| string validity_url = 6 [(validate.rules).string = {min_len: 1 prefix: "/"}]; | ||
|
|
||
| // Header that will be set if it is determined that the client can accept SXG (typically `accept: application/signed-exchange;v=b3) | ||
| // If not set, filter will default to: `x-client-can-accept-sxg` | ||
| string client_can_accept_sxg_header = 7 [ | ||
| (validate.rules).string = {well_known_regex: HTTP_HEADER_NAME strict: false ignore_empty: true} | ||
| ]; | ||
|
|
||
| // Header set by downstream service to signal that the response should be transformed to SXG If not set, | ||
| // filter will default to: `x-should-encode-sxg` | ||
| string should_encode_sxg_header = 8 [ | ||
| (validate.rules).string = {well_known_regex: HTTP_HEADER_NAME strict: false ignore_empty: true} | ||
| ]; | ||
|
|
||
| // Headers that will be stripped from the SXG document, by listing a prefix (i.e. `x-custom-` will cause | ||
| // all headers prefixed by `x-custom-` to be omitted from the SXG document) | ||
| repeated string header_prefix_filters = 9 [ | ||
| (validate.rules).repeated = {items {string {well_known_regex: HTTP_HEADER_NAME strict: false}}} | ||
| ]; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| # DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py. | ||
|
|
||
| load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package") | ||
|
|
||
| licenses(["notice"]) # Apache 2 | ||
|
|
||
| api_proto_package( | ||
| deps = [ | ||
| "//envoy/extensions/filters/http/sxg/v3alpha:pkg", | ||
| "//envoy/extensions/transport_sockets/tls/v4alpha:pkg", | ||
| "@com_github_cncf_udpa//udpa/annotations:pkg", | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package envoy.extensions.filters.http.sxg.v4alpha; | ||
|
|
||
| import "envoy/extensions/transport_sockets/tls/v4alpha/secret.proto"; | ||
|
|
||
| import "google/protobuf/duration.proto"; | ||
|
|
||
| import "udpa/annotations/status.proto"; | ||
| import "udpa/annotations/versioning.proto"; | ||
| import "validate/validate.proto"; | ||
|
|
||
| option java_package = "io.envoyproxy.envoy.extensions.filters.http.sxg.v4alpha"; | ||
| option java_outer_classname = "SxgProto"; | ||
| option java_multiple_files = true; | ||
| option (udpa.annotations.file_status).work_in_progress = true; | ||
| option (udpa.annotations.file_status).package_version_status = NEXT_MAJOR_VERSION_CANDIDATE; | ||
|
|
||
| // [#protodoc-title: Signed HTTP Exchange Filter] | ||
| // SXG :ref:`configuration overview <config_http_filters_sxg>`. | ||
| // [#extension: envoy.filters.http.sxg] | ||
|
|
||
| // [#next-free-field: 10] | ||
| message SXG { | ||
| option (udpa.annotations.versioning).previous_message_type = | ||
| "envoy.extensions.filters.http.sxg.v3alpha.SXG"; | ||
|
|
||
| // The SDS configuration for the public key data for the SSL certificate that will be used to sign the | ||
| // SXG response. | ||
| transport_sockets.tls.v4alpha.SdsSecretConfig certificate = 1; | ||
|
|
||
| // The SDS configuration for the private key data for the SSL certificate that will be used to sign the | ||
| // SXG response. | ||
| transport_sockets.tls.v4alpha.SdsSecretConfig private_key = 2; | ||
|
|
||
| // The duration for which the generated SXG package will be valid. Default is 604800s (7 days in seconds). | ||
| // Note that in order to account for clock skew, the timestamp will be backdated by a day. So, if duration | ||
| // is set to 7 days, that will be 7 days from 24 hours ago (6 days from now). Also note that while 6/7 days | ||
| // is appropriate for most content, if the downstream service is serving Javascript, or HTML with inline | ||
| // Javascript, 1 day (so, with backdated expiry, 2 days, or 172800 seconds) is more appropriate. | ||
| google.protobuf.Duration duration = 3; | ||
|
|
||
| // The SXG response payload is Merkle Integrity Content Encoding (MICE) encoded (specification is [here](https://datatracker.ietf.org/doc/html/draft-thomson-http-mice-03)) | ||
| // This value indicates the record size in the encoded payload. The default value is 4096. | ||
| uint64 mi_record_size = 4; | ||
|
|
||
| // The URI of certificate CBOR file published. Since it is required that the certificate CBOR file | ||
| // be served from the same domain as the SXG document, this should be a relative URI. | ||
| string cbor_url = 5 [(validate.rules).string = {min_len: 1 prefix: "/"}]; | ||
|
|
||
| // URL to retrieve validity data for signature, a CBOR map. See specification [here](https://tools.ietf.org/html/draft-yasskin-httpbis-origin-signed-exchanges-impl-00#section-3.6) | ||
| string validity_url = 6 [(validate.rules).string = {min_len: 1 prefix: "/"}]; | ||
|
|
||
| // Header that will be set if it is determined that the client can accept SXG (typically `accept: application/signed-exchange;v=b3) | ||
| // If not set, filter will default to: `x-client-can-accept-sxg` | ||
| string client_can_accept_sxg_header = 7 [ | ||
| (validate.rules).string = {well_known_regex: HTTP_HEADER_NAME strict: false ignore_empty: true} | ||
| ]; | ||
|
|
||
| // Header set by downstream service to signal that the response should be transformed to SXG If not set, | ||
| // filter will default to: `x-should-encode-sxg` | ||
| string should_encode_sxg_header = 8 [ | ||
| (validate.rules).string = {well_known_regex: HTTP_HEADER_NAME strict: false ignore_empty: true} | ||
| ]; | ||
|
|
||
| // Headers that will be stripped from the SXG document, by listing a prefix (i.e. `x-custom-` will cause | ||
| // all headers prefixed by `x-custom-` to be omitted from the SXG document) | ||
| repeated string header_prefix_filters = 9 [ | ||
| (validate.rules).repeated = {items {string {well_known_regex: HTTP_HEADER_NAME strict: false}}} | ||
| ]; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -43,6 +43,7 @@ HTTP filters | |
| router_filter | ||
| set_metadata_filter | ||
| squash_filter | ||
| sxg_filter | ||
| tap_filter | ||
| wasm_filter | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
|
|
||
| .. _config_http_filters_sxg: | ||
|
|
||
| SXG | ||
| ====== | ||
|
|
||
| * :ref:`v3 API reference <envoy_v3_api_msg_extensions.filters.http.sxg.v3alpha.SXG>` | ||
| * This filter should be configured with the name *envoy.filters.http.sxg*. | ||
|
Comment on lines
+7
to
+8
Member
There was a problem hiding this comment. Can you note this is in contrib builds only with a ref link to the contrib docs. If the squash filter docs don't have that can you fix that also? (I know I automated the other docs but I think for these manual docs it is probably missing.) |
||
|
|
||
| .. attention:: | ||
|
|
||
| The SXG filter is currently under active development. | ||
|
|
||
| This filter generates a Signed HTTP Exchange (SXG) package from a downstream web application. It uses `libsxg <https://github.com/google/libsxg/>`_ to perform the SXG packaging and signing, setting the Content-Type header to `application/signed-exchange;v=b3` and response body with the generated SXG document. | ||
|
|
||
| Transaction flow: | ||
|
|
||
| * check accept request header for whether client can accept SXG and set a flag. ``x-envoy-client-can-accept-sxg`` (or the header defined in ``client_can_accept_sxg_header``) will be set on the request | ||
| * If ``x-envoy-should-encode-sxg`` (or the header defined in ``should_encode_sxg_header``) is present in the response headers set a flag | ||
| * If both flags are set, buffer response body until stream end and then replace response body with generated the SXG | ||
|
|
||
| If there is an error generating the SXG package we fall back to the original HTML. | ||
|
|
||
| For more information on Signed HTTP Exchanges see `this doc <https://developers.google.com/web/updates/2018/11/signed-exchanges>`_. | ||
|
|
||
| Example configuration | ||
| --------------------- | ||
|
|
||
| The following is an example configuring the filter. | ||
|
|
||
| .. validated-code-block:: yaml | ||
| :type-name: envoy.extensions.filters.http.sxg.v3alpha.SXG | ||
|
|
||
| cbor_url: "/.sxg/cert.cbor" | ||
| validity_url: "/.sxg/validity.msg" | ||
| certificate: | ||
| name: certificate | ||
| sds_config: | ||
| path: "/etc/envoy/sxg-certificate.yaml" | ||
| private_key: | ||
| name: private_key | ||
| sds_config: | ||
| path: "/etc/envoy/sxg-private-key.yaml" | ||
| duration: 432000s | ||
| mi_record_size: 1024 | ||
| client_can_accept_sxg_header: "x-custom-accept-sxg" | ||
| should_encode_sxg_header: "x-custom-should-encode" | ||
| header_prefix_filters: | ||
|
cpapazian marked this conversation as resolved.
|
||
| - "x-foo-" | ||
| - "x-bar-" | ||
|
|
||
| Notes | ||
|
cpapazian marked this conversation as resolved.
|
||
| ----- | ||
|
|
||
| Instructions for generating a self-signed certificate and private key for testing can be found `here <https://github.com/WICG/webpackage/tree/master/go/signedexchange#creating-our-first-signed-exchange>`__ | ||
|
|
||
| Statistics | ||
| ---------- | ||
|
|
||
| The SXG filter outputs statistics in the *<stat_prefix>.sxg.* namespace. | ||
|
|
||
| .. csv-table:: | ||
| :header: Name, Type, Description | ||
| :widths: 1, 1, 2 | ||
|
|
||
| total_client_can_accept_sxg, Counter, Total requests where client passes valid Accept header for SXG documents. | ||
| total_should_sign, Counter, Total requests where downstream passes back header indicating Envoy should encocde document. | ||
| total_exceeded_max_payload_size, Counter, Total requests where response from downstream is to large. | ||
| total_signed_attempts, Counter, Total requests where SXG encoding is attempted. | ||
| total_signed_succeeded, Counter, Total requests where SXG encoding succeeds. | ||
| total_signed_failed, Counter, Total requests where SXG encoding fails. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the cert info need to come from SDS? We've recently been adding other options to
CommonTlsContext, such as theCertificateProviderInstancehooks. Can we leverage that here somehow instead of hard-coding use of SDS?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, it does not need to be SDS. The version of this that we are running currently, uses an internal platform for secrets. I was using the Oauth2 filter extension as a template, and it uses SDS. Happy to use
CertificateProviderInstanceor similar if that's preferred. Do you have an example I could look at?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the rationale for departing from SDS?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@envoyproxy/api-shepherds to get some guidance on SDS vs less SDS for similar use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, the way the protos are currently set up, we don't seem to have a single reusable proto that represents a "certificate source" that can be configured for any of the various supported mechanisms. The infrastructure here is a little weak.
For now, I guess this is okay as-is, just because we don't have anything better to offer. Just keep in mind that we will probably need to update this in the future as this infrastructure improves.