diff --git a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-apache.asciidoc b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-apache.asciidoc index 6922c1d02..938962aec 100644 --- a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-apache.asciidoc +++ b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-apache.asciidoc @@ -5,11 +5,88 @@ Apache ++++ // tag::apache-jobs[] -These {anomaly-job} wizards appear in {kib} if you use -{filebeat-ref}/index.html[{filebeat}] to ship access logs from your -https://httpd.apache.org/[Apache] HTTP servers to {es} and store it using fields -and data types from the Elastic Common Schema (ECS). For more details, see the -{dfeed} and job definitions in +These {anomaly-job} wizards appear in {kib} if you use the Apache integration in +{fleet} or you use {filebeat} to ship access logs from your +https://httpd.apache.org/[Apache] HTTP servers to {es}. The jobs assume that you +use fields and data types from the Elastic Common Schema (ECS). + +[[apache-access-logs]] +== Apache access logs + +These {anomaly-jobs} find unusual activity in HTTP access logs. + +For more details, see the {dfeed} and job definitions in +https://github.com/elastic/integrations/blob/{branch}/packages/apache/kibana/ml_module/apache-Logs-ml.json[GitHub]. +Note that these jobs are available in {kib} only if data exists that matches the +{dfeed} query. + +low_request_rate_apache:: +Detects low request rates. + +Job details::: + +* Analyzes request rates (using the <>). + +Required {beats} or {agent} integrations::: + +* Apache integration + +source_ip_request_rate_apache:: +Detects unusual source IPs. + +Job details::: + +* Analyzes request rates (using the <>) +relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: + +* Apache integration + +source_ip_url_count_apache:: +Detects unusual source IPs. + +Job details::: + +* Analyzes distinct counts of URLs (using the +<> on the `url.original` +field) relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: + +* Apache integration + +status_code_rate_apache:: +Detects unusual status code rates. + +Job details::: + +* Analyzes request rates (using the <>) split by +status code (`partition_field_name` is `http.response.status_code`). + +Required {beats} or {agent} integrations::: + +* Apache integration + +visitor_rate_apache:: +Detects unusual visitor rates. + +Job details::: + +* Analyzes request rates using the <>. + +Required {beats} or {agent} integrations::: + +* Apache integration + +[[apache-access-logs-filebeat]] +== Apache access logs ({filebeat}) + +These legacy {anomaly-jobs} find unusual activity in HTTP access logs. For the +latest versions, install the Apache integration in {fleet}; see +<>. + +For more details, see the {dfeed} and job definitions in https://github.com/elastic/kibana/tree/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/apache_ecs/ml[GitHub]. These configurations are only available if data exists that matches the @@ -17,37 +94,63 @@ recognizer query specified in the https://github.com/elastic/kibana/blob/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/apache_ecs/manifest.json#L8[manifest file]. low_request_rate_ecs:: +Detects low request rates. + +Job details::: -* For HTTP web access logs where `event.dataset` is `apache.access`. -* Models the event rate of HTTP requests. -* Detects unusually low counts of HTTP requests compared to the previous event - rate. +* Analyzes request rates (using the <>). + +Required {beats} or {agent} integrations::: + +* {filebeat} source_ip_request_rate_ecs:: +Detects unusual source IPs. + +Job details::: -* For HTTP web access logs where `event.dataset` is `apache.access`. -* Models the event rate of HTTP requests by source IP. -* Detects source IPs with unusually high request rates in the HTTP access log - compared to the previous rate. +* Analyzes request rates (using the <>) +relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: + +* {filebeat} source_ip_url_count_ecs:: +Detects unusal source IPs. + +Job details::: + +* Analyzes distinct counts of URLs (using the +<> on the `url.original` +field) relative to all the source IPs (`over_field_name` is `source.address`). -* For HTTP web access logs where `event.dataset` is `apache.access`. -* Models the event rate of HTTP requests by source IP. -* Detects source IPs with unusually high distinct count of URLs in the HTTP -access log. +Required {beats} or {agent} integrations::: + +* {filebeat} status_code_rate_ecs:: -* For HTTP web access logs where `event.dataset` is `apache.access`. -* Models the occurrences of HTTP response status codes. -* Detects unusual status code rates in the HTTP access log compared to previous - rates. +Detects unusual status code rates. + +Job details::: + +* Analyzes request rates (using the <>) split by +status code (`partition_field_name` is `http.response.status_code`). + +Required {beats} or {agent} integrations::: + +* {filebeat} visitor_rate_ecs:: +Detects unusual visitor rates. + +Job details::: + +* Analyzes request rates using the <>. + +Required {beats} or {agent} integrations::: + +* {filebeat} -* For HTTP web access logs where `event.dataset` is `apache.access`. -* Models visitor rates. -* Detects unusual visitor rates in the HTTP access log compared to previous - rates. // end::apache-jobs[] \ No newline at end of file diff --git a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-nginx.asciidoc b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-nginx.asciidoc index 92ac78717..e9e1c60fc 100644 --- a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-nginx.asciidoc +++ b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-nginx.asciidoc @@ -6,10 +6,89 @@ ++++ // tag::nginx-jobs[] -These {anomaly-job} wizards appear in {kib} if you use {filebeat} to ship access -logs from your http://nginx.org/[Nginx] HTTP servers to {es} and store it using -fields and datatypes from the Elastic Common Schema (ECS). For more details, see -the {dfeed} and job definitions in + +These {anomaly-job} wizards appear in {kib} if you use the Nginx integration in +{fleet} or you use {filebeat} to ship access logs from your +http://nginx.org/[Nginx] HTTP servers to {es}. The jobs assume that you use +fields and data types from the Elastic Common Schema (ECS). + +[[nginx-access-logs]] +== Nginx access logs + +These {anomaly-jobs} find unusual activity in HTTP access logs. + +For more details, see the {dfeed} and job definitions in +https://github.com/elastic/integrations/blob/{branch}/packages/nginx/kibana/ml_module/nginx-Logs-ml.json[GitHub]. +Note that these jobs are available in {kib} only if data exists that matches the +{dfeed} query. + +low_request_rate_nginx:: +Detects low request rates. + +Job details::: + +* Analyzes request rates (using the <>). + +Required {beats} or {agent} integrations::: + +* Nginx integration + +source_ip_request_rate_nginx:: +Detects unusual source IPs. + +Job details::: + +* Analyzes request rates (using the <>) +relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: + +* Nginx integration + +source_ip_url_count_nginx:: +Detects unusual source IPs. + +Job details::: + +* Analyzes distinct counts of URLs (using the +<> on the `url.original` +field) relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: + +* Nginx integration + +status_code_rate_nginx:: +Detects unusual status code rates. + +Job details::: + +* Analyzes request rates (using the <>) split by +status code (`partition_field_name` is `http.response.status_code`). + +Required {beats} or {agent} integrations::: + +* Nginx integration + +visitor_rate_nginx:: +Detects unusual visitor rates. + +Job details::: + +* Analyzes request rates using the <>. + +Required {beats} or {agent} integrations::: + +* Nginx integration + +[[nginx-access-logs-filebeat]] +== Nginx access logs ({filebeat}) + +These legacy {anomaly-jobs} find unusual activity in HTTP access logs. For the +latest versions, install the Nginx integration in {fleet}; see +<>. + +For more details, see the {dfeed} and job definitions in https://github.com/elastic/kibana/tree/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules/nginx_ecs/ml[GitHub]. These configurations are only available if data exists that matches the @@ -18,38 +97,62 @@ https://github.com/elastic/kibana/blob/{branch}/x-pack/plugins/ml/server/models/ low_request_rate_ecs:: +Detects low request rates. + +Job details::: -* For HTTP web access logs where `event.dataset` is `nginx.access`. -* Models the event rate of http requests. -* Detects unusually low counts of HTTP requests compared to the previous event - rate. +* Analyzes request rates (using the <>). + +Required {beats} or {agent} integrations::: + +* {filebeat} source_ip_request_rate_ecs:: +Detects unusual source IPs. + +Job details::: + +* Analyzes request rates (using the <>) +relative to all the source IPs (`over_field_name` is `source.address`). -* For HTTP web access logs where `event.dataset` is `nginx.access`. -* Models the event rate of HTTP requests by source IP. -* Detects source IPs with unusually high request rates in the HTTP access log - compared to the previous rate. +Required {beats} or {agent} integrations::: + +* {filebeat} source_ip_url_count_ecs:: +Detects unusual source IPs. + +Job details::: + +* Analyzes distinct counts of URLs (using the +<> on the `url.original` +field) relative to all the source IPs (`over_field_name` is `source.address`). + +Required {beats} or {agent} integrations::: -* For HTTP web access logs where `event.dataset` is `nginx.access`. -* Models the event rate of HTTP requests by source IP. -* Detects source IPs with unusually high distinct count of URLs in the HTTP - access log. +* {filebeat} status_code_rate_ecs:: +Detects unusual status code rates. -* For HTTP web access logs where `event.dataset` is `nginx.access`. -* Models the occurrences of HTTP response status codes. -* Detects unusual status code rates in the HTTP access log compared to previous - rates. +Job details::: + +* Analyzes request rates (using the <>) split by +status code (`partition_field_name` is `http.response.status_code`). + +Required {beats} or {agent} integrations::: + +* {filebeat} visitor_rate_ecs:: +Detects unusual visitor rates. + +Job details::: + +* Analyzes request rates using the <>. + +Required {beats} or {agent} integrations::: -* For HTTP web access logs where `event.dataset` is `nginx.access`. -* Models visitor rates. -* Detects unusual visitor rates in the HTTP access log compared to previous - rates. +* {filebeat} -// end::nginx-jobs[] \ No newline at end of file +// end::nginx-jobs[]