-
Notifications
You must be signed in to change notification settings - Fork 25.6k
[DOCS] Adds data frame analytics APIs to the ML APIs #43875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
5997037
[DOCS] Adds data frame analytics APIs to the ML APIs.
szabosteve d2e3edd
[DOCS] Addresses build errors
lcawl b7b0d49
[DOCS] More minor edits
lcawl 29467fd
[DOCS] Adds missing delimiter
lcawl beb9f12
Fixes the file extension.
szabosteve 1eda71b
[DOCS] Unifies terminology.
szabosteve 93cc7db
[DOCS] Addresses feedback.
szabosteve 361a17b
[DOCS] Fixes typos.
szabosteve efde71f
[DOCS] Fixes references.
szabosteve 955e373
[DOCS] Adds description section to the PUT API.
szabosteve 7593cfc
[DOCS] Minor changes.
szabosteve 0454112
[DOCS] Fixes gradle checks
lcawl b69bd31
[DOCS] Fixes typo
lcawl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| [role="xpack"] | ||
| [testenv="platinum"] | ||
| [[delete-dfanalytics]] | ||
| === Delete {dfanalytics-jobs} API | ||
| [subs="attributes"] | ||
| ++++ | ||
| <titleabbrev>Delete {dfanalytics-jobs}</titleabbrev> | ||
| ++++ | ||
|
|
||
| experimental[] | ||
|
|
||
| Deletes an existing {dfanalytics-job}. | ||
|
|
||
| [[ml-delete-dfanalytics-request]] | ||
| ==== {api-request-title} | ||
|
|
||
| `DELETE _ml/data_frame/analytics/<data_frame_analytics_id>` | ||
|
|
||
| [[ml-delete-dfanalytics-prereq]] | ||
| ==== {api-prereq-title} | ||
|
|
||
| * You must have `machine_learning_admin` built-in role to use this API. For more | ||
| information, see {stack-ov}/security-privileges.html[Security privileges] and | ||
| {stack-ov}/built-in-roles.html[Built-in roles]. | ||
|
|
||
| [[ml-delete-dfanalytics-path-params]] | ||
| ==== {api-path-parms-title} | ||
|
|
||
| `<data_frame_analytics_id>` (Required):: | ||
| (string) Identifier for the {dfanalytics-job} you want to delete. | ||
|
|
||
| [[ml-delete-dfanalytics-example]] | ||
| ==== {api-examples-title} | ||
|
|
||
| The following example deletes the `loganalytics` {dfanalytics-job}: | ||
|
|
||
| [source,js] | ||
| -------------------------------------------------- | ||
| DELETE _ml/data_frame/analytics/loganalytics | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| // TEST | ||
|
|
||
| The API returns the following result: | ||
|
|
||
| [source,js] | ||
| ---- | ||
| { | ||
| "acknowledged" : true | ||
| } | ||
| ---- | ||
| // TESTRESPONSE | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| [role="xpack"] | ||
| [testenv="platinum"] | ||
| [[evaluate-dfanalytics]] | ||
| === Evaluate {dfanalytics} API | ||
|
|
||
| [subs="attributes"] | ||
| ++++ | ||
| <titleabbrev>Evaluate {dfanalytics}</titleabbrev> | ||
| ++++ | ||
|
|
||
| experimental[] | ||
|
|
||
| Evaluates the executed analysis on an index that is already annotated with a | ||
szabosteve marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| field that contains the results of the analytics (the `ground truth`) for each | ||
| {dataframe} row. Evaluation is typically done via calculating a set of metrics | ||
| that capture various aspects of the quality of the results over the data for | ||
| which we have the `ground truth`. For different types of analyses different | ||
| metrics are suitable. This API packages together commonly used metrics for | ||
| various analyses. | ||
|
|
||
| [[ml-evaluate-dfanalytics-request]] | ||
| ==== {api-request-title} | ||
|
|
||
| `POST _ml/data_frame/_evaluate` | ||
|
|
||
| [[ml-evaluate-dfanalytics-prereq]] | ||
| ==== {api-prereq-title} | ||
|
|
||
| * You must have `monitor_ml` privilege to use this API. For more | ||
| information, see {stack-ov}/security-privileges.html[Security privileges] and | ||
| {stack-ov}/built-in-roles.html[Built-in roles]. | ||
|
|
||
| [[ml-evaluate-dfanalytics-request-body]] | ||
| ==== {api-request-body-title} | ||
|
|
||
| `index` (Required):: | ||
| (object) Defines the `index` in which the evaluation will be performed. | ||
|
|
||
| `evaluation` (Required):: | ||
| (object) Defines the type of evaluation you want to perform. For example: | ||
| `binary_soft_classification`. | ||
| See evaluate API resources. | ||
|
|
||
| [[ml-evaluate-dfanalytics-example]] | ||
| ==== {api-examples-title} | ||
|
|
||
| [source,js] | ||
| -------------------------------------------------- | ||
| POST _ml/data_frame/_evaluate | ||
|
|
||
| { | ||
| "index": "my_analytics_dest_index" | ||
| "evaluation": { | ||
| "binary_soft_classification": { | ||
| "actual_field": "is_outlier" | ||
| "predicted_probability_field": "ml.outlier_score" | ||
| } | ||
| } | ||
| } | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| // TEST | ||
|
|
||
| The API returns the following results: | ||
|
|
||
| [source,js] | ||
| ---- | ||
| { | ||
| "binary_soft_classification": { | ||
| "auc_roc": { | ||
| "score": 0.92584757746414444 | ||
| }, | ||
| "confusion_matrix": { | ||
| "0.25": { | ||
| "tp": 5, | ||
| "fp": 9, | ||
| "tn": 204, | ||
| "fn": 5 | ||
| }, | ||
| "0.5": { | ||
| "tp": 1, | ||
| "fp": 5, | ||
| "tn": 208, | ||
| "fn": 9 | ||
| }, | ||
| "0.75": { | ||
| "tp": 0, | ||
| "fp": 4, | ||
| "tn": 209, | ||
| "fn": 10 | ||
| } | ||
| }, | ||
| "precision": { | ||
| "0.25": 0.35714285714285715, | ||
| "0.5": 0.16666666666666666, | ||
| "0.75": 0 | ||
| }, | ||
| "recall": { | ||
| "0.25": 0.5, | ||
| "0.5": 0.1, | ||
| "0.75": 0 | ||
| } | ||
| } | ||
| } | ||
| ---- | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| [role="xpack"] | ||
| [testenv="platinum"] | ||
| [[get-dfanalytics-stats]] | ||
| === Get {dfanalytics-jobs} statistics API | ||
| [subs="attributes"] | ||
| ++++ | ||
| <titleabbrev>Get {dfanalytics-jobs} stats</titleabbrev> | ||
| ++++ | ||
|
|
||
| experimental[] | ||
|
|
||
| Retrieves usage information for {dfanalytics-jobs}. | ||
|
|
||
| [[ml-get-dfanalytics-stats-request]] | ||
| ==== {api-request-title} | ||
|
|
||
| `GET _ml/data_frame/analytics/<data_frame_analytics_id>/_stats` + | ||
|
|
||
| `GET _ml/data_frame/analytics/<data_frame_analytics_id>,<data_frame_analytics_id>/_stats` + | ||
|
|
||
| `GET _ml/data_frame/analytics/_stats` + | ||
|
|
||
| `GET _ml/data_frame/analytics/_all/_stats` + | ||
|
|
||
| `GET _ml/data_frame/analytics/*/_stats` | ||
|
|
||
| [[ml-get-dfanalytics-stats-prereq]] | ||
| ==== {api-prereq-title} | ||
|
|
||
| * You must have `monitor_ml` privilege to use this API. For more | ||
| information, see {stack-ov}/security-privileges.html[Security privileges] and | ||
| {stack-ov}/built-in-roles.html[Built-in roles]. | ||
|
|
||
| [[ml-get-dfanalytics-stats-path-params]] | ||
| ==== {api-path-parms-title} | ||
|
|
||
| `<data_frame_analytics_id>` (Optional):: | ||
| (string) Identifier for the {dfanalytics-job}. If you do not specify one of | ||
| these options, the API returns information for the first hundred | ||
| {dfanalytics-jobs}. | ||
|
|
||
| `allow_no_match` (Optional) | ||
| (boolean) If `false` and the `data_frame_analytics_id` does not match any | ||
| {dfanalytics-job} an error will be returned. The default value is `true`. | ||
|
|
||
| [[ml-get-dfanalytics-stats-query-params]] | ||
| ==== {api-query-parms-title} | ||
|
|
||
| `from` (Optional):: | ||
| (integer) Skips the specified number of {dfanalytics-jobs}. The default value | ||
| is `0`. | ||
|
|
||
| `size` (Optional):: | ||
| (integer) Specifies the maximum number of {dfanalytics-jobs} to obtain. The | ||
| default value is `100`. | ||
|
|
||
| [discrete] | ||
| [[ml-get-dfanalytics-stats-response-body]] | ||
| ==== {api-response-body-title} | ||
|
|
||
| The API returns the following information: | ||
|
|
||
| `data_frame_analytics`:: | ||
| (array) An array of statistics objects for {dfanalytics-jobs}, which are | ||
| sorted by the `id` value in ascending order. | ||
|
|
||
| [[ml-get-dfanalytics-stats-example]] | ||
| ==== {api-examples-title} | ||
|
|
||
| [source,js] | ||
| -------------------------------------------------- | ||
| GET _ml/data_frame/analytics/loganalytics/_stats | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| // TEST | ||
|
|
||
| The API returns the following results: | ||
|
|
||
| [source,js] | ||
| ---- | ||
| { | ||
| "count": 1, | ||
| "data_frame_analytics": [ | ||
| { | ||
| "id": "loganalytics", | ||
| "state": "stopped" | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
| // TESTRESPONSE |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| [role="xpack"] | ||
| [testenv="platinum"] | ||
| [[get-dfanalytics]] | ||
| === Get {dfanalytics-jobs} API | ||
| [subs="attributes"] | ||
| ++++ | ||
| <titleabbrev>Get {dfanalytics-jobs}</titleabbrev> | ||
| ++++ | ||
|
|
||
| experimental[] | ||
|
|
||
| Retrieves configuration information for {dfanalytics-jobs}. | ||
|
|
||
| [[ml-get-dfanalytics-request]] | ||
| ==== {api-request-title} | ||
|
|
||
| `GET _ml/data_frame/analytics/<data_frame_analytics_id>` + | ||
|
|
||
| `GET _ml/data_frame/analytics/<data_frame_analytics_id>,<data_frame_analytics_id>` + | ||
|
|
||
| `GET _ml/data_frame/analytics/` + | ||
|
|
||
| `GET _ml/data_frame/analytics/_all` | ||
|
|
||
| [[ml-get-dfanalytics-prereq]] | ||
| ==== {api-prereq-title} | ||
|
|
||
| * You must have `monitor_ml` privilege to use this API. For more | ||
| information, see {stack-ov}/security-privileges.html[Security privileges] and | ||
| {stack-ov}/built-in-roles.html[Built-in roles]. | ||
|
|
||
| [[ml-get-dfanalytics-desc]] | ||
| ==== {api-description-title} | ||
|
|
||
| You can get information for multiple {dfanalytics-jobs} in a single API request | ||
| by using a comma-separated list of {dfanalytics-jobs} or a wildcard expression. | ||
| You can get information for all {dfanalytics-jobs} by using _all, by specifying | ||
| `*` as the `<data_frame_analytics_id>`, or by omitting the | ||
| `<data_frame_analytics_id>`. | ||
|
|
||
| [[ml-get-dfanalytics-path-params]] | ||
| ==== {api-path-parms-title} | ||
|
|
||
| `<data_frame_analytics_id>` (Optional):: | ||
| (string) Identifier for the {dfanalytics-job}. If you do not specify one of | ||
| these options, the API returns information for the first hundred | ||
| {dfanalytics-jobs}. | ||
|
|
||
| `allow_no_match` (Optional) | ||
| (boolean) If `false` and the `data_frame_analytics_id` does not match any | ||
| {dfanalytics-job} an error will be returned. The default value is `true`. | ||
|
|
||
| [[ml-get-dfanalytics-query-params]] | ||
| ==== {api-query-parms-title} | ||
|
|
||
| `from` (Optional):: | ||
| (integer) Skips the specified number of {dfanalytics-jobs}. The default value | ||
| is `0`. | ||
|
|
||
| `size` (Optional):: | ||
| (integer) Specifies the maximum number of {dfanalytics-jobs} to obtain. The | ||
| default value is `100`. | ||
|
|
||
| [[ml-get-dfanalytics-example]] | ||
| ==== {api-examples-title} | ||
|
|
||
| The following example gets configuration information for the `loganalytics` | ||
| {dfanalytics-job}: | ||
|
|
||
| [source,js] | ||
| -------------------------------------------------- | ||
| GET _ml/data_frame/analytics/loganalytics | ||
| -------------------------------------------------- | ||
| // CONSOLE | ||
| // TEST[skip:set up kibana samples] | ||
|
|
||
| The API returns the following results: | ||
|
|
||
| [source,js] | ||
| ---- | ||
| { | ||
| "count": 1, | ||
| "data_frame_analytics": [ | ||
| { | ||
| "id": "loganalytics", | ||
| "source": { | ||
| "index": "logdata", | ||
| "query": { | ||
| "match_all": {} | ||
| } | ||
| }, | ||
| "dest": { | ||
| "index": "logdata_out", | ||
| "results_field": "ml" | ||
| }, | ||
| "analysis": { | ||
| "outlier_detection": {} | ||
| }, | ||
| "model_memory_limit": "1gb" | ||
| } | ||
| ] | ||
| } | ||
| ---- | ||
| // TESTRESPONSE |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.