Skip to content

JSON Schema: HTTP Connection Manager access log #412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ccaraman merged 10 commits into from
Feb 6, 2017
Merged

JSON Schema: HTTP Connection Manager access log #412

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
705f949
put away for now
ccaraman Feb 2, 2017
53f5848
clean up
ccaraman Feb 2, 2017
ba00be1
cleaning
ccaraman Feb 2, 2017
1c84dc9
spacing
ccaraman Feb 2, 2017
37380b1
finally works
ccaraman Feb 3, 2017
28118a6
newline
ccaraman Feb 3, 2017
803ba83
int range
ccaraman Feb 3, 2017
30d5ca3
599
ccaraman Feb 3, 2017
e9cc037
random sampling
ccaraman Feb 4, 2017
532a6dc
make travis work again
ccaraman Feb 4, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 27 additions & 2 deletions docs/configuration/http_conn_man/access_log.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ format
.. _config_http_conn_man_access_log_filter_param:

filter
*(optional, string)* :ref:`Filter <config_http_con_manager_access_log_filters>` which is used to
*(optional, object)* :ref:`Filter <config_http_con_manager_access_log_filters>` which is used to
determine if the access log needs to be written.

.. _config_http_con_manager_access_log_format:
Expand Down Expand Up @@ -175,7 +175,7 @@ Duration
Filters on total request duration in milliseconds.

op
*(required, string)* Comparison operator. Currently *>=* is the only supported operator.
*(required, string)* Comparison operator. Currently *>=* and *=* are the only supported operators.

value
*(required, integer)* Default value to compare against if runtime values is not available.
Expand Down Expand Up @@ -212,6 +212,31 @@ Traceable
Filters for requests that are traceable. See the :ref:`tracing overview <arch_overview_tracing>` for
more information on how a request becomes traceable.


.. _config_http_con_manager_access_log_filters_runtime:

Runtime
^^^^^^^^^
.. code-block:: json

{
"filter": {
"type": "runtime",
"key" : "..."
}
}

Filters for random sampling of requests. Sampling pivots on the header
:ref:`x-request-id<config_http_conn_man_headers_x-request-id>` being present. If
:ref:`x-request-id<config_http_conn_man_headers_x-request-id>` is present, the filter will
consistently sample across multiple hosts based on the runtime key value and the value extracted
from :ref:`x-request-id<config_http_conn_man_headers_x-request-id>`. If it is missing, the
filter will randomly sample based on the runtime key value.

key
*(required, string)* Runtime key to get the percentage of requests to be sampled.
This runtime control is specified in the range 0-100 and defaults to 0.

And
^^^

Expand Down
3 changes: 2 additions & 1 deletion docs/configuration/http_conn_man/headers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,8 @@ is one of the few areas where a thin client library is needed to perform this du
is out of scope for this documentation. If *x-request-id* is propagated across all hosts, the
following features are available:

* Stable :ref:`access logging <config_http_conn_man_access_log>` via the sampling filter.
* Stable :ref:`access logging <config_http_conn_man_access_log>` via the
:ref:`runtime filter<config_http_con_manager_access_log_filters_runtime>`.
* Stable tracing when performing random sampling via the :ref:`tracing.random_sampling
<config_http_conn_man_runtime_random_sampling>` runtime setting or via forced tracing using the
:ref:`config_http_conn_man_headers_x-envoy-force-trace` and
Expand Down
124 changes: 117 additions & 7 deletions source/common/json/config_schemas.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,104 @@ const std::string Json::Schema::HTTP_CONN_NETWORK_FILTER_SCHEMA(R"EOF(
{
"$schema": "http://json-schema.org/schema#",
"definitions" : {
"access_log": {
"status_code" : {
"type" : "object",
"properties" : {
"path" : {"type" : "string"},
"format" : {"type" : "string"},
"filter" : {"type" : "object"}
"type" : {
"type" : "string",
"enum" : ["status_code"]
},
"op" : {
"type" : "string",
"enum" : [">=", "="]
},
"value" : {
"type" : "integer",
"minimum" : 0,
"maximum" : 599
},
"runtime_key" : {"type" : "string"}
},
"required" : ["type", "op", "value"],
"additionalProperties" : false
},
"duration" : {
"type" : "object",
"properties" : {
"type" : {
"type" : "string",
"enum" : ["duration"]
},
"op" : {
"type" : "string",
"enum" : [">=", "="]
},
"value" : {
"type" : "integer",
"minimum" : 0
},
"runtime_key" : {"type" : "string"}
},
"required" : ["path"],
"required" : ["type", "op", "value"],
"additionalProperties" : false
},
"not_healthcheck" : {
"type" : "object",
"properties" : {
"type" : {
"type" : "string",
"enum" : ["not_healthcheck"]
}
},
"required" : ["type"],
"additionalProperties" : false
},
"traceable_request" : {
"type" : "object",
"properties" : {
"type" : {
"type" : "string",
"enum" : ["traceable_request"]
}
},
"required" : ["type"],
"additionalProperties" : false
},
"runtime" : {
"type" : "object",
"properties" : {
"type" : {
"type" : "string",
"enum" : ["runtime"]
},
"key" : {"type" : "string"}
},
"required" : ["type", "key"],
"additionalProperties" : false
},
"logical_filter" : {
"type" : "object",
"properties" : {
"type" : {
"type" : "string",
"enum" : ["logical_and", "logical_or"]
},
"filters" : {
"type" : "array",
"minItems" : 2,
"items" : {
"oneOf" : [
{"$ref" : "#/definitions/status_code"},
{"$ref" : "#/definitions/duration"},
{"$ref" : "#/definitions/not_healthcheck"},
{"$ref" : "#/definitions/logical_filter"},
{"$ref" : "#/definitions/traceable_request"},
{"$ref" : "#/definitions/runtime"}
]
}
}
},
"required" : ["type", "filters"],
"additionalProperties" : false
},
"tracing" : {
Expand Down Expand Up @@ -133,7 +223,23 @@ const std::string Json::Schema::HTTP_CONN_NETWORK_FILTER_SCHEMA(R"EOF(
"type" : "array",
"items" : {
"type" : "object",
"properties" : {"$ref" : "#/definitions/access_log"}
"properties" : {
"path" : {"type" : "string"},
"format" : {"type" : "string"},
"filter" : {
"type" : "object",
"oneOf" : [
{"$ref" : "#/definitions/not_healthcheck"},
{"$ref" : "#/definitions/status_code"},
{"$ref" : "#/definitions/duration"},
{"$ref" : "#/definitions/traceable_request"},
{"$ref" : "#/definitions/runtime"},
{"$ref" : "#/definitions/logical_filter"}
]
}
},
"required" : ["path"],
"additionalProperties" : false
}
},
"use_remote_address" : {"type" : "boolean"},
Expand Down Expand Up @@ -487,7 +593,11 @@ const std::string Json::Schema::FAULT_HTTP_FILTER_SCHEMA(R"EOF(
"minimum" : 0,
"maximum" : 100
},
"http_status" : {"type" : "integer"}
"http_status" : {
"type" : "integer",
"minimum" : 0,
"maximum" : 599
}
},
"required" : ["abort_percent", "http_status"],
"additionalProperties" : false
Expand Down
38 changes: 30 additions & 8 deletions test/config/integration/server.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down Expand Up @@ -107,10 +118,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down
38 changes: 30 additions & 8 deletions test/config/integration/server_http2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
Copy link
Member

@RomanDzhabarov RomanDzhabarov Feb 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you use '=' also in integration test configs, so that it'll have additional coverage

Copy link
Contributor Author

@ccaraman ccaraman Feb 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already coverage for '=' within access_log_impl_tests.cc

Copy link
Member

@RomanDzhabarov RomanDzhabarov Feb 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup it's tested, just wanted extra test of that here, either way

"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down Expand Up @@ -88,10 +99,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down
57 changes: 45 additions & 12 deletions test/config/integration/server_http2_upstream.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down Expand Up @@ -82,10 +93,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down Expand Up @@ -157,10 +179,21 @@
"access_log": [
{
"path": "/dev/null",
"filters": [
{"type": "status_code", "op": ">=", "value": 500},
{"type": "duration", "op": ">=", "value": 1000000}
]
"filter" : {
"type": "logical_or",
"filters": [
{
"type": "status_code",
"op": ">=",
"value": 500
},
{
"type": "duration",
"op": ">=",
"value": 1000000
}
]
}
},
{
"path": "/dev/null"
Expand Down
Loading