Skip to content

[ML] Anomaly detection job custom_settings improvements#102099

Merged
jgowdyelastic merged 10 commits intoelastic:masterfrom
jgowdyelastic:job-custom_settings-improvements
Jun 18, 2021
Merged

[ML] Anomaly detection job custom_settings improvements#102099
jgowdyelastic merged 10 commits intoelastic:masterfrom
jgowdyelastic:job-custom_settings-improvements

Conversation

@jgowdyelastic
Copy link
Member

@jgowdyelastic jgowdyelastic commented Jun 14, 2021

Adds Job tags and a general Custom settings to the list of properties listed in a job's expanded row:

image

Adds the ability to search for job_tags with job_tags:<tag>:<value>
e.g. job_tags:"euid:8015" or job_tags:(euid:8015 or euid:8016) to match multiple tags.
image

Fixes #101877

cc @randomuserid

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@jgowdyelastic jgowdyelastic requested a review from a team as a code owner June 15, 2021 14:19
@jgowdyelastic jgowdyelastic added the auto-backport Deprecated - use backport:version if exact versions are needed label Jun 15, 2021
@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

Copy link
Member

@spong spong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked out locally and verified ML Job Settings UI functions as intended -- Security Solution changes LGTM!

@peteharverson
Copy link
Contributor

peteharverson commented Jun 16, 2021

Testing this I found an issue with the reloading of expanded rows when the search changes. If you expand a row, edit the search, and then set it back to its previous value, the previously expanded row shows the loading icon which never disappears. Not related to searching for job_tags - also happens with the existing job ID / description based search.

job_search_bug

This also happens if you expand a row, go to the next page, and then back to the starting page - there is a loading indicator displayed for the row that was previously expanded.

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although I did find an existing bug in the search functionality with expanded rows.

@jgowdyelastic
Copy link
Member Author

@elasticmachine merge upstream

@jgowdyelastic
Copy link
Member Author

@peteharverson row expansion issue fixed in 8a107a8

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
ml 270 272 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
ml 5.9MB 5.9MB +1.9KB
securitySolution 6.9MB 6.9MB +11.0B
total +1.9KB
Unknown metric groups

API count

id before after diff
ml 274 276 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

@jgowdyelastic jgowdyelastic merged commit 0ef1c3d into elastic:master Jun 18, 2021
@jgowdyelastic jgowdyelastic deleted the job-custom_settings-improvements branch June 18, 2021 08:31
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 18, 2021
* [ML] Anomaly detection job custom_settings improvements

* filter improvements

* translations

* fixing types

* fixing tests

* one more test fix

* fixing bug with expanded row

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
7.x

This backport PR will be merged automatically after passing CI.

jloleysens added a commit to jloleysens/kibana that referenced this pull request Jun 18, 2021
…ets-tab

* 'master' of github.com:elastic/kibana: (93 commits)
  [ML] Remove blank job definition as it is unused and out-of-sync with Elasticsearch (elastic#102506)
  [Lens] Fix wrong error detection on transition to Top values operation (elastic#102384)
  [ML] Anomaly detection job custom_settings improvements (elastic#102099)
  [Cases] Route: Get all alerts attach to a case (elastic#101878)
  Fixes wrong list exception type when creating endpoint event filters list (elastic#102522)
  remove search bar that's not working yet (elastic#102550)
  Migrated Ingest Node Pipeline Functional Tests to use test_user (elastic#102409)
  [Maps] clean up feature editing name space to avoid conflicts with layer settings editing (elastic#102516)
  [canvas] Refactor Storybook from bespoke to standard configuration (elastic#101962)
  [Security Solution] adds wrapSequences method (RAC) (elastic#102106)
  [FTR] Stabilize SSLP functional tests (elastic#102553)
  [K8] Added `Inter` font files for new theme (elastic#102359)
  [Workplace Search] Convert Groups pages to new page template (elastic#102449)
  [DOC] Add experimental disclaimer to rollup jobs (elastic#95624)
  [Security Solution][Endpoint] Suppress some of the jest console.error noise created by endpoint list middelware (elastic#102535)
  [Fleet] Improve performance of Fleet setup (elastic#102219)
  [Alerting] Add event log entry when a rule starts executing (elastic#102001)
  [Fleet] Update docker image of registry used in integration tests (elastic#101911)
  [Asset Management] Osquery telemetry updates (elastic#100754)
  Converts saved object tagging to new management layout (elastic#102284)
  ...

# Conflicts:
#	x-pack/plugins/fleet/kibana.json
kibanamachine added a commit that referenced this pull request Jun 18, 2021
…02609)

* [ML] Anomaly detection job custom_settings improvements

* filter improvements

* translations

* fixing types

* fixing tests

* one more test fix

* fixing bug with expanded row

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: James Gowdy <jgowdy@elastic.co>
jloleysens added a commit to jloleysens/kibana that referenced this pull request Jun 21, 2021
…-png-pdf-report-type

* 'master' of github.com:elastic/kibana: (447 commits)
  skip flaky suite (elastic#102366)
  [Security Solution][Endpoint][Host Isolation] Isolation status badge from alert details (elastic#102274)
  Add email connector info for Elastic Cloud (elastic#91363)
  [Workplace Search] remove or replace xs props for text on source connect view (elastic#102663)
  Do not double register dashboard url generator (elastic#102599)
  [TSVB] Replaces EuiCodeEditor 👉 Monaco editor  (elastic#100684)
  [Discover] Update kibana.json adding owner and description (elastic#102292)
  [Exploratory View] Mobile experience (elastic#99565)
  chore(NA): moving @kbn/ui-shared-deps into bazel (elastic#101669)
  [TSVB] Index pattern select field disappear in Annotation tab (elastic#102314)
  [Security Solution][Endpoint][Host Isolation] Fixes bug where host isolation/unisolation works from alert details (elastic#102581)
  TSVB visualizations with no timefield do not render after upgrading from 7.12.1 to 7.13.0 (elastic#102494)
  [Logs UI] Add `event.original` fallback to message reconstruction rules (elastic#102236)
  [ML] Remove blank job definition as it is unused and out-of-sync with Elasticsearch (elastic#102506)
  [Lens] Fix wrong error detection on transition to Top values operation (elastic#102384)
  [ML] Anomaly detection job custom_settings improvements (elastic#102099)
  [Cases] Route: Get all alerts attach to a case (elastic#101878)
  Fixes wrong list exception type when creating endpoint event filters list (elastic#102522)
  remove search bar that's not working yet (elastic#102550)
  Migrated Ingest Node Pipeline Functional Tests to use test_user (elastic#102409)
  ...

# Conflicts:
#	x-pack/plugins/reporting/public/share_context_menu/register_pdf_png_reporting.tsx
@randomuserid
Copy link
Contributor

randomuserid commented Jun 23, 2021

Looks great, all of these searches evaluate true;

job_tags:(euid:8015 or euid:8016)
job_tags:(event.category:authentication)
job_tags:(maturity:experimental)
job_tags:(updated_date:5/12/2021)
job_tags:(version:1)

It looks like a search for author does not evaluate true if a user puts their Twitter handle in the field like this;

job_tags:(author:@randomuserid)

Do we need to escape the @ char somehow? If this isn't going to work, initially, I can ask people to use their GitHub name, if they have one, or their common name, if they don't.

@peteharverson
Copy link
Contributor

Created #104706 to fix the issue for searches using the @ character.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Deprecated - use backport:version if exact versions are needed Feature:Anomaly Detection ML anomaly detection :ml release_note:enhancement review v7.14.0 v8.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ML] Add support for searchable GUID anomaly detection job tag

6 participants