Skip to content
Closed
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
899ec21
Init work with configuration proposition
Aug 12, 2019
6da905d
Added missing dependency
Aug 31, 2019
8eb6f82
Merge branch 'master' into local_reply_mapper
Aug 31, 2019
9bf0e6b
Without passed config
Aug 31, 2019
fc3c545
WIP config
Sep 2, 2019
c9a686e
Response flags added
Sep 3, 2019
4213f2e
Applied changes from code review
Sep 4, 2019
f8df3ae
repeated match
Sep 4, 2019
dac4695
Merge branch 'master' into local_reply_mapper
Sep 7, 2019
052aeeb
WIP local reply formatter with access log reuse
Sep 7, 2019
1499746
Merge branch 'master' into local_reply_mapper
Sep 13, 2019
8f022b6
Added mapper config
Sep 15, 2019
419a59c
Changed to access log filter
Sep 18, 2019
468308a
Merge branch 'master' into local_reply_mapper
Sep 19, 2019
1cce878
Cleanup and initial work docs
Sep 19, 2019
1efc2e3
Cleanup and initial work docs
Sep 19, 2019
7c75f6c
added comments
Sep 20, 2019
2670d0f
Added possibility to define custom format for each mapper
Sep 26, 2019
955bf79
Merge branch 'master' into local_reply_mapper
Sep 26, 2019
cd31254
Extracted new format StringOrJson
Sep 27, 2019
a77c82f
Removed docs and fixed format. I will provide docs file with implemen…
Oct 1, 2019
20abff0
Merge branch 'master' into local_reply_mapper
Oct 2, 2019
9c8d757
Added more docs
Oct 2, 2019
cc55bb4
Merge branch 'master' into local_reply_mapper
Oct 4, 2019
7784130
Code review fix
Oct 4, 2019
31f3cbf
Merge branch 'master' into local_reply_mapper
Oct 10, 2019
5eadeca
Fix build
Oct 11, 2019
199dd43
Merge branch 'master' into local_reply_mapper
Oct 11, 2019
0aee07a
Generated v3 files
Oct 11, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import "validate/validate.proto";
// [#protodoc-title: HTTP connection manager]
// HTTP connection manager :ref:`configuration overview <config_http_conn_man>`.

// [#comment:next free field: 35]
// [#comment:next free field: 36]
message HttpConnectionManager {
enum CodecType {

Expand Down Expand Up @@ -453,6 +453,40 @@ message HttpConnectionManager {
// with `prefix` match set to `/dir`. Defaults to `false`. Note that slash merging is not part of
// `HTTP spec <https://tools.ietf.org/html/rfc3986>` and is provided for convenience.
bool merge_slashes = 33;

message LocalReplyConfig {
// Configuration of list of mappers which allows to filter and change local response.
// Code iterate through mappers until first match of filter.
repeated ResponseMapper mapper = 1;

// Allows to define custom format of local reply.
oneof reply_format {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something like this belongs in envoy.type, as it comes up in other places, e.g. access logs. It's basically a StringOrJson type.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find this type. Maybe it should be added in v3?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting we start the trend by adding it in this PR :) Up to you, at least leave a [#next-major-version:...] comment if not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added new format. You can check if it looks ok. I haven't changed it in v2 accesslog config because it isn't compatible with current acceslog format. Example:

format:
  string_format: 

// Response body plain text format.
string format = 2;

// Response body json format as dictionary.
google.protobuf.Struct json_format = 3;
}
};

// Configuration of local reply returned by Envoy. Allows to specify mapping and format of response.
LocalReplyConfig local_reply_config = 35;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should LocalReplyConfig be one-per ResponseMapper? What's the tradeoff here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think LocalReplyConfig should be one by http_conn_manager, but we can define many Mappers. The tradeoff is that you can define only one format per manager. So if you want to define custom format for different response it won't be possible.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we think that there might be possibly a use case here for multiple custom format, I'd just allow it in the API; the implementation can probably deal with this without too much complexity. I.e. let's be really sure that one format will be sufficient forever if we take the existing design.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added possibility to define format for each mapper. I think about implementation: if mapper format is defined then use mapper format else use format from config (or default if not defined)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the way you added it, it seems we could punt this until later, if you really want to keep the top-level reply_format. So, I'd either rollback that change or remove the top-level reply_format; I gather you want it though to reduce verbosity if you have many rewriters?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I thought it might be good idea to define one format at top-level and allow to define custom one for each mapper if someone want different format. I can also remove top level mapper but if someone want to change the format of all responses it will require definition of mapper without filter/rewriter.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, given that, maybe just completely ignore the above thread and rollback the per-rewriter bits, since they can be added in a non-breaking way later additively. It'll simplify the implementation work at your end. Sorry about the confusion, the PR clarified how this would look, thanks.

}

message ResponseMapper {
// Filter is used to determine if the response should be changed.
envoy.config.filter.accesslog.v2.AccessLogFilter filter = 1 [(validate.rules).message.required = true];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, if this works I think adding #next-major-version comment in AccessLogFilter to rename it more generically.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, this sounds great.


// Rewriter defines which values in local reply should be changed.
ResponseRewriter rewriter = 2 [(validate.rules).message.required = true];
}

// Configuration of new value for matched local response.
message ResponseRewriter {
// Status code for matched response.
google.protobuf.UInt32Value status_code = 1;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this just be uint32?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be good to keep this field optional if in future someone want to add e.g: headers, body.


// TODO: add support for GRPC rewrite.
}

message Rds {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ HTTP connection manager
stats
runtime
rds
local_reply
103 changes: 103 additions & 0 deletions docs/root/configuration/http/http_conn_man/local_reply.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
.. _config_http_conn_man_local_reply:

Local reply modification
========================

The :ref:`HTTP connection manager <arch_overview_http_conn_man>` supports modification of local reply which is response returned by Envoy itself
rather than response from cluster.

Features:

* :ref:`Local reply content modification<config_http_conn_man_local_reply_modification>`.
* :ref:`Local reply format modification<config_http_conn_man_local_reply_format>`.

.. _config_http_conn_man_local_reply_modification:

Local reply content modification
--------------------------------

There is support for modification of local replies. You can specify list of :ref:`mappers <envoy_api_field_config.filter.network.http_connection_manager.v2.HttpConnectionManager.LocalReplyConfig.mapper>` which contains
pairs of :ref:`filter <envoy_api_msg_config.filter.accesslog.v2.AccessLogFilter>` and :ref:`rewriter <envoy_api_msg_config.filter.network.http_connection_manager.v2.ResponseRewriter>`. Both elements in pair has to be
specified. If more than one pair is defined then first matching is used.

Example how to change status code when local reply contains any of these response flags:

.. code-block:: yaml

mapper:
filter:
response_flag_filter:
flags:
- LH
- UH
rewriter:
status_code: 504

.. _config_http_conn_man_local_reply_format:

Local reply format modification
-------------------------------

Local reply format contains command operators that extract the relevant data and insert it.
They support two formats: :ref:`format strings <config_http_conn_man_local_reply_format_string>` and
:ref:`"format dictionaries" <config_http_conn_man_local_reply_dictionaries>`. In both cases, the :ref:`command operators <config_http_conn_man_local_reply_command_operators>`
are used to extract the relevant data, which is then inserted into the specified reply format.
Only one reply format may be specified at the time.

.. _config_http_conn_man_local_reply_format_string:

Format Strings
--------------

Format strings are plain strings, specified using the ``format`` key. They may contain
either :ref:`command operators <config_http_conn_man_local_reply_command_operators>` or other characters interpreted as a plain string.
The access log formatter does not make any assumptions about a new line separator, so one
has to specified as part of the format string.

.. code-block:: none

%RESP_BODY% %RESPONSE_CODE% %RESPONSE_FLAGS% "My custom response"

Example of custom Envoy local reply format:

.. code-block:: none

upstream connect error or disconnect/reset before headers. reset reason: connection failure 204 UH My custom response


If format isn't specified then :ref:`default format <config_http_conn_man_local_reply_default_format>` is used.

.. _config_http_conn_man_local_reply_default_format:

Default Format String
---------------------

If custom format string is not specified, Envoy uses the following default format:

.. code-block:: none

%RESP_BODY%

Example of the default local reply format:

.. code-block:: none

upstream connect error or disconnect/reset before headers. reset reason: connection failure

.. _config_http_conn_man_local_reply_dictionaries:

Format Dictionaries
-------------------

Format dictionaries are dictionaries that specify a structured local reply output format,
specified using the ``json_format`` key. This allows response to be returned in a structured format
such as JSON.

More can be found in :ref:`configuration <config_access_log_format_dictionaries>`.

.. _config_http_conn_man_local_reply_command_operators:

Command Operators
-----------------

Local reply format reuse :ref:`access log operators <config_access_log_command_operators>`, so more information can be found there.
2 changes: 2 additions & 0 deletions docs/root/configuration/observability/access_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ Format dictionaries have the following restrictions:

* The dictionary must map strings to strings (specifically, strings to command operators). Nesting is not currently supported.

.. _config_access_log_command_operators:

Command Operators
-----------------

Expand Down