From b8375a1f73d1144b2aa7360acdd37f2ace87d414 Mon Sep 17 00:00:00 2001 From: hejianpeng Date: Mon, 28 Nov 2022 11:30:55 +0800 Subject: [PATCH 1/5] add buf-gen-docs --- Makefile.core.mk | 3 +++ buf.gen.yaml | 8 ++++++++ 2 files changed, 11 insertions(+) create mode 100644 buf.gen.yaml diff --git a/Makefile.core.mk b/Makefile.core.mk index d4955af1998..21b24f3bf4c 100644 --- a/Makefile.core.mk +++ b/Makefile.core.mk @@ -111,6 +111,9 @@ check_wasm: build_wasm build_envoy clean: @bazel clean +gen-extensions-doc: + buf generate --path extensions/ + gen: @scripts/gen-testdata.sh diff --git a/buf.gen.yaml b/buf.gen.yaml new file mode 100644 index 00000000000..70e8095de5d --- /dev/null +++ b/buf.gen.yaml @@ -0,0 +1,8 @@ +# buf.gen.yaml sets up the generation configuration for all of our plugins. +# Note: buf does not allow multi roots that are within each other; as a result, the common-protos folders are +# symlinked into the top level directory. +version: v1 +plugins: +- name: docs + out: . + opt: warnings=false,per_file=true,mode=html_fragment_with_front_matter From 7cfaa66340d34cb224e1bad7471c93f837266b5a Mon Sep 17 00:00:00 2001 From: hejianpeng Date: Mon, 28 Nov 2022 11:32:17 +0800 Subject: [PATCH 2/5] update extensions doc --- .../v1alpha1/access_log_policy_config.pb.html | 18 +- extensions/attributegen/config.pb.html | 34 +-- extensions/metadata_exchange/config.pb.html | 3 +- .../stackdriver_plugin_config.pb.html | 193 ++++++++++++------ extensions/stats/config.pb.html | 126 ++++++------ 5 files changed, 208 insertions(+), 166 deletions(-) diff --git a/extensions/access_log_policy/config/v1alpha1/access_log_policy_config.pb.html b/extensions/access_log_policy/config/v1alpha1/access_log_policy_config.pb.html index a3be641c7ca..ca506b80d59 100644 --- a/extensions/access_log_policy/config/v1alpha1/access_log_policy_config.pb.html +++ b/extensions/access_log_policy/config/v1alpha1/access_log_policy_config.pb.html @@ -8,14 +8,16 @@ number_of_entries: 1 ---

Accesslog Policy plugin is a stateful http log sampler. -It decides whether a request is logged based on the following rules. - 1. All requests resulting in errors are logged. - 2. First successful request within logwindowduration from a specific - source ip (source principal) is logged. -The plugin records its decision in the istio.accesslogpolicy attribute with +It decides whether a request is logged based on the following rules.

+
    +
  1. All requests resulting in errors are logged.
    2. +
  2. First successful request within log_window_duration from a specific +source ip (source principal) is logged. +The plugin records its decision in the istio.access_log_policy attribute with a value of “no”. A downstream plugin may honor the the attribute. For example, Stackdriver plugin will not produce an access log entry if this -attribute is set.

    +attribute is set.
    3. +

AccessLogPolicyConfig

@@ -32,7 +34,7 @@

AccessLogPolicyConfig

-log_window_duration +logWindowDuration Duration

Optional. Allows specifying logging window for successful requests. @@ -44,7 +46,7 @@

AccessLogPolicyConfig

-max_client_cache_size +maxClientCacheSize int32

Optional. Allows specifying max client cache size. diff --git a/extensions/attributegen/config.pb.html b/extensions/attributegen/config.pb.html index 961bc4141f5..912fce6a148 100644 --- a/extensions/attributegen/config.pb.html +++ b/extensions/attributegen/config.pb.html @@ -8,15 +8,13 @@ weight: 20 number_of_entries: 3 --- -

AttributeGen plugin uses builtin attributes +

AttributeGen plugin uses [builtin attributes] +(https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) as inputs and produces new attributes that can be used by downstream plugins.

-

The following is an example of a configuration that produces one attribute named istio_operationId using request.url_path and request.method.

-

{{}} {{}}

- 
{
   "attributes": [
     {
@@ -44,20 +42,15 @@
 }
-

{{}} {{}}

-

If the Stats plugin runs after AttributeGen, it can use istio_operationId to populate a dimension on a metric.

-

The following is an example of response codes being mapped into a smaller number of response classes as the istio_responseClass attribute. For example, all response codes in 200s are mapped to 2xx.

-

{{}} {{}}

- 
{
   "attributes": [
     {
@@ -97,11 +90,9 @@
 }
-

{{}} {{}}

- -

If multiple AttributeGene configurations produce the same attribute, the +

If multiple AttributeGen configurations produce the same attribute, the result of the last configuration will be visible to downstream filters.

PluginConfig

@@ -160,28 +151,23 @@

AttributeGeneration

-output_attribute +outputAttribute string

The name of the attribute that is populated on a successful match. An attribute name SHOULD NOT contain a .. You may use underscores for namespacing instead.

-

Example: istio_operationId

-

istio_ attribute namespace is reserved by Istio.

-

AttributeGeneration may fail to evaluate when an attribute is not available. For example, response.code may not be available when a request ends abruptly. When attribute generation fails, it will not populate the attribute.

-

If the generated attribute is used by an authz plugin, it should account for the possibility that the attribute may be missing. Use has(attribute_name) function to check for presence of an attribute before using its value, and provide appropriate defaults. For example the following is a safe use of response.code

-

has(response.code)?response.code:200

@@ -226,13 +212,11 @@

Match

The condition is a CEL expression -that may use builtin attributes.

- +that may use [builtin attributes] +(https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes).

Example:

-

{{}} {{}}

- 
   {
      "value": "GetBook",
      "condition":
@@ -240,23 +224,17 @@ 
Match

      && request.method == 'GET'"
    },
-

Note: CEL uses re2 regex library. Use anchors {^, $} to ensure that the regex evaluates efficiently.

-

Note: request.url_path is normalized and stripped of query params.

-

a Read only operation on books

- 
{ "value": "ReadOnlyBooks",
   "condition": "request.url_path.startsWith('/books/') &&
   in(request.method, ['GET', 'HEAD'])"}
-

{{}} {{}}

-

An empty condition evaluates to true and should be used to provide a default value.

diff --git a/extensions/metadata_exchange/config.pb.html b/extensions/metadata_exchange/config.pb.html index 1ea4ab5b2d1..b5ffa83eace 100644 --- a/extensions/metadata_exchange/config.pb.html +++ b/extensions/metadata_exchange/config.pb.html @@ -20,7 +20,7 @@

PluginConfig

-max_peer_cache_size +maxPeerCacheSize UInt32Value

maximum size of the peer metadata cache. @@ -38,7 +38,6 @@

PluginConfig

google.protobuf.UInt32Value

Wrapper message for uint32.

-

The JSON representation for UInt32Value is JSON number.

diff --git a/extensions/stackdriver/config/v1alpha1/stackdriver_plugin_config.pb.html b/extensions/stackdriver/config/v1alpha1/stackdriver_plugin_config.pb.html index 1f5f2505db8..f3678c97acc 100644 --- a/extensions/stackdriver/config/v1alpha1/stackdriver_plugin_config.pb.html +++ b/extensions/stackdriver/config/v1alpha1/stackdriver_plugin_config.pb.html @@ -5,12 +5,12 @@ layout: protoc-gen-docs generator: protoc-gen-docs weight: 20 -number_of_entries: 3 +number_of_entries: 4 ---

CustomConfig

Custom instance configuration overrides. -Provides a way to customize metrics/logs.

+Provides a way to customize logs.

@@ -40,7 +40,7 @@

CustomConfig

PluginConfig

-

next id: 15

+

next id: 17

@@ -53,7 +53,7 @@

PluginConfig

- + - + - + - + - - - - - - - - - - - - - + - + - + - - - + + + - - - + + + - - - + + + - + @@ -221,7 +194,7 @@

PluginConfig

- + + + + + + + - + + + + + + + + + + + + + + + + + + + + + + +
max_log_batch_size_in_bytesmaxLogBatchSizeInBytes int32

Optional. Allows configuration of the size of the LogWrite request. The @@ -66,7 +66,7 @@

PluginConfig
log_report_durationlogReportDuration Duration

Optional. Allows configuration of the time between calls out to the @@ -80,7 +80,7 @@

PluginConfig
enable_audit_logenableAuditLog bool

Optional. Controls whether to export audit log.

@@ -91,49 +91,20 @@

PluginConfig
destination_service_namedestinationServiceName string

Optional. FQDN of destination service that the request routed to, e.g. productpage.default.svc.cluster.local. If not provided, request host header will be used instead

-		 -No -
enable_mesh_edges_reportingbool -

Optional. Controls whether or not to export mesh edges to a mesh edges -service. This is disabled by default.

- -		 -No -
mesh_edges_reporting_durationDuration -

Optional. Allows configuration of the time between calls out to the mesh -edges service to report NEW edges. The minimum configurable duration is -10s. NOTE: This option ONLY configures the intermediate reporting of -novel edges. Once every 10m, all edges observed in that 10m window are -reported and the local cache is cleared. -The default duration is 1m. Any value greater than 10m will result in -reporting every 10m.

- 		No
max_peer_cache_sizemaxPeerCacheSize int32

maximum size of the peer metadata cache. @@ -146,7 +117,7 @@

PluginConfig
disable_host_header_fallbackdisableHostHeaderFallback bool

Optional: Disable using host header as a fallback if destination service is @@ -159,7 +130,7 @@

PluginConfig
max_edges_batch_sizemaxEdgesBatchSize int32

Optional. Allows configuration of the number of traffic assertions to batch @@ -170,35 +141,36 @@

PluginConfig

No
disable_http_size_metricsbool
enableLogCompressionBoolValue -

Optional. Allows disabling of reporting of the request and response size -metrics for HTTP traffic. Defaults to false (request and response size -metrics are enabled).

+

Optional. Allows enabling log compression for stackdriver access logs.

 No
enable_log_compressionBoolValue
accessLoggingAccessLogging -

Optional. Allows enabling log compression for stackdriver access logs.

+

Optional. Controls what type of logs to export.

 No
access_loggingAccessLogging
accessLoggingFilterExpressionstring -

Optional. Controls what type of logs to export..

+

CEL expression for filtering access logging. If the expression evaluates +to true, an access log entry will be generated. Otherwise, no access log +entry will be generated. +NOTE: Audit logs ignore configured filters.

 @@ -206,13 +178,14 @@

PluginConfig
custom_log_configcustomLogConfig CustomConfig

(Optional) Collection of tag names and tag expressions to include in the logs. Conflicts are resolved by the tag name by overriding previously supplied values. Does not apply to audit logs. -See https://istio.io/latest/docs/tasks/observability/metrics/customize-metrics/#use-expressions-for-values +See +https://istio.io/latest/docs/tasks/observability/metrics/customize-metrics/#use-expressions-for-values for more details about the expression language.
metric_expiry_durationmetricExpiryDuration Duration

Optional. Controls the metric expiry duration. If a metric time series is @@ -230,18 +203,121 @@

PluginConfig

series will never be expired. This option is useful to avoid unbounded metric label explodes proxy memory.

+		 +No +
metricsOverridesmap<string, MetricsOverride> +

Optional. Allows altering metrics behavior. +Metric names for specifying overloads drop the istio.io/service prefix. +Examples: server/request_count, client/roundtrip_latencies

+ 		No
disable_server_access_loggingdisableServerAccessLogging bool

Optional. Controls whether to export server access log. This is deprecated in favor of AccessLogging enum.

+		 +No +
enableMeshEdgesReportingbool +

Optional. Controls whether or not to export mesh edges to a mesh edges +service. This is disabled by default. +Deprecated – Mesh edge reporting is no longer supported and this setting +is no-op.

+ +		 +No +
meshEdgesReportingDurationDuration +

Optional. Allows configuration of the time between calls out to the mesh +edges service to report NEW edges. The minimum configurable duration is +10s. NOTE: This option ONLY configures the intermediate reporting of +novel edges. Once every 10m, all edges observed in that 10m window are +reported and the local cache is cleared. +The default duration is 1m. Any value greater than 10m will result in +reporting every 10m. +Deprecated – Mesh edge reporting is no longer supported and this setting +is no-op.

+ +		 +No +
disableHttpSizeMetricsbool +

Optional. Allows disabling of reporting of the request and response size +metrics for HTTP traffic. Defaults to false (request and response size +metrics are enabled). +Deprecated – use metrics_overrides instead. +if metrics_overrides is used, this value will be ignored.

+ +		 +No +
+
+

MetricsOverride

+
+

Provides behavior modifications for Cloud Monitoring metrics.

+ + + + + + + + + + + + + + + + + + + + + diff --git a/extensions/stats/config.pb.html b/extensions/stats/config.pb.html index cd17aa80b05..e44d675664d 100644 --- a/extensions/stats/config.pb.html +++ b/extensions/stats/config.pb.html @@ -5,7 +5,7 @@ layout: protoc-gen-docs generator: protoc-gen-docs weight: 20 -number_of_entries: 4 +number_of_entries: 5 ---

MetricConfig

@@ -52,7 +52,7 @@

MetricConfig

- + + + + + + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - + + + + + + +
FieldTypeDescriptionRequired
dropbool +

Optional. If true, no data for the associated metric will be collected or +exported.

+ +		 +No +
tagOverridesmap<string, string> +

Optional. Maps tag names to value expressions that will be used at +reporting time. If the tag name does not match a well-known tag for the +istio Cloud Monitoring metrics, the configuration will have no effect.

+ 		No @@ -280,8 +356,7 @@

PluginConfig.AccessLogging

ERRORS_ONLY

All error logs. This is currently only available for outbound/client side -logs. A request is classified as error when status>=400 or -response_flag != "-"

+logs. A request is classified as error when status>=400 or response_flag != "-"
tags_to_removetagsToRemove string[]

(Optional) A list of tags to remove.

@@ -68,6 +68,18 @@

MetricConfig

NOT IMPLEMENTED. (Optional) Conditional enabling the override.

+		 +No +
dropbool +

(Optional) If this is set to true, the metric(s) selected by this +configuration will not be generated or reported.

+ 		No @@ -136,73 +148,8 @@

PluginConfig

debugbool -

next id: 7 -The following settings should be rarely used. -Enable debug for this filter. -DEPRECATED.

- -		 -No -
max_peer_cache_sizeint32 -

maximum size of the peer metadata cache. -A long lived proxy that connects with many transient peers can build up a -large cache. To turn off the cache, set this field to a negative value. -DEPRECATED.

- -		 -No -
stat_prefixstring -

prefix to add to stats emitted by the plugin. -DEPRECATED.

- -		 -No -
field_separatorstring -

Stats api squashes dimensions in a single string. -The squashed string is parsed at prometheus scrape time to recover -dimensions. The following 2 fields set the field and value separators {key: -value} –> key{valueseparator}value{fieldseparator}

- -		 -No -
value_separatorstring -

default: “==”

- -		 -No -
disable_host_header_fallbackdisableHostHeaderFallback bool

Optional: Disable using host header as a fallback if destination service is @@ -215,7 +162,7 @@

PluginConfig
tcp_reporting_durationtcpReportingDuration Duration

Optional. Allows configuration of the time between calls out to for TCP @@ -243,6 +190,17 @@

PluginConfig

Metric definitions.

+		 +No +
reporterReporter +

Proxy deployment type.

+ 		No @@ -279,3 +237,33 @@

MetricType
+

Reporter

+
+

Specifies the proxy deployment type.

+ + + + + + + + + + + + + + + + + + +
NameDescription
UNSPECIFIED +

Default value is inferred from the listener direction, as either client or +server sidecar.

+ +
SERVER_GATEWAY +

Shared server gateway, e.g. “waypoint”.

+ +
+
From b0f6a210aacee9b0a2b16940572bb77d349e8dae Mon Sep 17 00:00:00 2001 From: hejianpeng Date: Mon, 28 Nov 2022 11:41:03 +0800 Subject: [PATCH 3/5] fix --- Makefile.core.mk | 1 + extensions/attributegen/config.pb.html | 3 +-- extensions/attributegen/config.proto | 6 ++---- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/Makefile.core.mk b/Makefile.core.mk index 21b24f3bf4c..6ddb0f06416 100644 --- a/Makefile.core.mk +++ b/Makefile.core.mk @@ -111,6 +111,7 @@ check_wasm: build_wasm build_envoy clean: @bazel clean +.PHONY: gen-extensions-doc gen-extensions-doc: buf generate --path extensions/ diff --git a/extensions/attributegen/config.pb.html b/extensions/attributegen/config.pb.html index 912fce6a148..7b5f4a0b1fc 100644 --- a/extensions/attributegen/config.pb.html +++ b/extensions/attributegen/config.pb.html @@ -8,8 +8,7 @@ weight: 20 number_of_entries: 3 --- -

AttributeGen plugin uses [builtin attributes] -(https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) +

AttributeGen plugin uses builtin attributes as inputs and produces new attributes that can be used by downstream plugins.

The following is an example of a configuration that produces one attribute named istio_operationId using request.url_path and request.method.

diff --git a/extensions/attributegen/config.proto b/extensions/attributegen/config.proto index 1a9f9985a6e..491c8d5a2dd 100644 --- a/extensions/attributegen/config.proto +++ b/extensions/attributegen/config.proto @@ -23,8 +23,7 @@ syntax = "proto3"; // $weight: 20 // clang-format on -// AttributeGen plugin uses [builtin attributes] -// (https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) +// AttributeGen plugin uses [builtin attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) // as inputs and produces new attributes that can be used by downstream plugins. // // The following is an example of a configuration that produces one attribute @@ -170,8 +169,7 @@ message AttributeGeneration { message Match { // The condition is a [CEL // expression](https://github.com/google/cel-spec/blob/master/doc/langdef.md) - // that may use [builtin attributes] - // (https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes). + // that may use [builtin attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes). // // Example: // From c38c4ecc8d30cd164301a0e66d98e0937ab76cdc Mon Sep 17 00:00:00 2001 From: hejianpeng Date: Mon, 28 Nov 2022 11:42:26 +0800 Subject: [PATCH 4/5] fix format --- extensions/attributegen/config.pb.html | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/extensions/attributegen/config.pb.html b/extensions/attributegen/config.pb.html index 7b5f4a0b1fc..b545c1d51e5 100644 --- a/extensions/attributegen/config.pb.html +++ b/extensions/attributegen/config.pb.html @@ -211,8 +211,7 @@

Match

The condition is a CEL expression -that may use [builtin attributes] -(https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes).

+that may use builtin attributes.

Example:

{{}} {{}}

From b0d14b04f2f06cd69178f17504923711bff84b56 Mon Sep 17 00:00:00 2001 From: hejianpeng Date: Mon, 28 Nov 2022 11:55:47 +0800 Subject: [PATCH 5/5] fix lint --- extensions/attributegen/config.pb.html | 6 ++++-- extensions/attributegen/config.proto | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/extensions/attributegen/config.pb.html b/extensions/attributegen/config.pb.html index b545c1d51e5..7f9ab99e8d2 100644 --- a/extensions/attributegen/config.pb.html +++ b/extensions/attributegen/config.pb.html @@ -8,7 +8,8 @@ weight: 20 number_of_entries: 3 --- -

AttributeGen plugin uses builtin attributes +

AttributeGen plugin uses builtin +attributes as inputs and produces new attributes that can be used by downstream plugins.

The following is an example of a configuration that produces one attribute named istio_operationId using request.url_path and request.method.

@@ -211,7 +212,8 @@

Match

The condition is a CEL expression -that may use builtin attributes.

+that may use builtin +attributes.

Example:

{{}} {{}}

diff --git a/extensions/attributegen/config.proto b/extensions/attributegen/config.proto index 491c8d5a2dd..e0c9b04a900 100644 --- a/extensions/attributegen/config.proto +++ b/extensions/attributegen/config.proto @@ -23,7 +23,8 @@ syntax = "proto3"; // $weight: 20 // clang-format on -// AttributeGen plugin uses [builtin attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) +// AttributeGen plugin uses [builtin +// attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) // as inputs and produces new attributes that can be used by downstream plugins. // // The following is an example of a configuration that produces one attribute @@ -169,7 +170,8 @@ message AttributeGeneration { message Match { // The condition is a [CEL // expression](https://github.com/google/cel-spec/blob/master/doc/langdef.md) - // that may use [builtin attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes). + // that may use [builtin + // attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#attributes). // // Example: //