Skip to content

[ML] Add a "verbose" option to the data frame analytics stats endpoint #59125

@droberts195

Description

@droberts195

The data frame analytics stats endpoint can return extremely verbose information that is only useful if it is plotted on a chart. For example:

Screenshot 2020-04-07 at 12 52 24

For REST API clients that do not have the ability to plot the fold_values on a chart the information is just clutter that makes it hard to see other elements of the stats.

To avoid this problem we should have a ?verbose option on the REST endpoint for data frame analytics stats. (This does not need to be implemented in the Java clients as they can always retrieve the whole object and the user just doesn't call the methods that return verbose information if they don't want it.)

An example of a parameter that only affects the X-Content representation of the results is ?explain on the find_file_structure endpoint. This doesn't go into the request but instead gets passed in the params argument to the toXContent methods that render the response in JSON so they can decide not to print certain things. So ?verbose for data frame analytics stats can be done similarly.

One complication is that these same toXContent methods also create the JSON when the stats are indexed. In this case we need to keep all the fields. One way to make this work would be to make them write all the fields by default and only skip the detail when the verbose parameter was explicitly false, but default verbose to false if not specified in the REST handler. Another way would be to have a FOR_INTERNAL_STORAGE parameter that we pass from the places where we're persisting to an index.

The final question is exactly which fields to omit when ?verbose is not specified. Certainly fold_values, but maybe there are others. Please have a look and propose something that seems reasonable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions