[ML] Add warnings for actions for managed Anomaly detection jobs and Transforms by qn895 · Pull Request #122305 · elastic/kibana

qn895 · 2022-01-05T00:27:34Z

Summary

This PR addresses #120631 adds warning modals when users modify or delete one or more Anomaly detection jobs or Transforms that are managed by other solutions. If the job is not managed, these modals will not be visible.

Anomaly detection

Delete

Edit

Stop

Start

Close

Reset

Transforms

Delete

Start

Stop

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

...ns/ml/public/application/jobs/jobs_list/components/confirm_modals/edit_job_confirm_modal.tsx

...ublic/application/jobs/jobs_list/components/confirm_modals/start_datafeeds_confirm_modal.tsx

...public/application/jobs/jobs_list/components/confirm_modals/stop_datafeeds_confirm_modal.tsx

...lugins/ml/public/application/jobs/jobs_list/components/delete_job_modal/delete_job_modal.tsx

...rm/public/app/sections/transform_management/components/action_delete/delete_action_modal.tsx

...form/public/app/sections/transform_management/components/action_start/start_action_modal.tsx

elasticmachine · 2022-01-06T15:00:51Z

Pinging @elastic/ml-ui (:ml)

jgowdyelastic · 2022-01-06T16:26:02Z

A special formatter is needed to correctly show the managed boolean value in the jobs list.

Actually, there are other booleans that are missing. This can be done in a follow up.

jgowdyelastic · 2022-01-06T16:42:14Z

We probably want to add the managed warning to the delete modal when the user doesn't have access to all of the spaces the job belongs in.

And when the user has access to all spaces

jgowdyelastic · 2022-01-06T16:50:17Z

I'm not sure about having these warnings in new modals.
They could be added as callouts to existing modals or flyouts and would be less intrusive.
Plus from a code point of view, the callout could just be a single component that takes a job and decides whether or not to display itself.

E.g. at the top of the edit jobs flyout

or in the delete modal

For the stop datafeed option we would need a new modal, but it could contain the same callout.

Thoughts @peteharverson @qn895 ?

jgowdyelastic · 2022-01-06T17:00:57Z

It would be helpful to have a tooltip on the managed label on the jobs list, to explain what a managed job is.

jgowdyelastic · 2022-01-06T17:56:36Z

Do we want to strip out the managed flag when cloning a job? Currently it's left in.
My initial thought was to remove it, as it won't be the exact job that the non-ml plugin is looking for.
However some plugins (security) look for groups and list all jobs in a known group. They may want the ability to clone a job and it be treated exactly like the original.

sophiec20 · 2022-01-06T18:00:27Z

For the examples we have, the managed job has to have a specific job_id .. therefore I do not believe we should clone the managed tag.

...ublic/application/jobs/jobs_list/components/confirm_modals/start_datafeeds_confirm_modal.tsx

jgowdyelastic · 2022-01-14T09:32:33Z

...lugins/ml/public/application/jobs/jobs_list/components/delete_job_modal/delete_job_modal.tsx

-                jobId: jobIds[0],
-              }}
-            />
+            {


nit, these braces aren't needed

Removed here 352559c

jgowdyelastic · 2022-01-14T09:33:14Z

...k/plugins/ml/public/application/jobs/jobs_list/components/edit_job_flyout/edit_job_flyout.js

        jobClosed,
      } = this.state;

+      console.log('job', job, isManagedJob(job));


this console.log should probably be removed.

Removed here 352559c

jgowdyelastic · 2022-01-14T09:49:19Z

...public/application/jobs/jobs_list/components/confirm_modals/stop_datafeeds_confirm_modal.tsx

+import { i18n } from '@kbn/i18n';
+import { MlSummaryJob } from '../../../../../../common/types/anomaly_detection_jobs';
+import { isManagedJob } from '../../../jobs_utils';
+// @ts-ignore


we have a utils.d.ts that stopDatafeeds could be added to to avoid this @ts-ignore

Updated here af963d1

jgowdyelastic · 2022-01-14T10:08:29Z

x-pack/plugins/ml/server/models/job_service/jobs.ts

+        // 'Managed' means job is preconfigured and deployed by other solutions
+        // we should not clone the setting
+        if (job.custom_settings?.managed === true) {
+          job.custom_settings.managed = false;


My draft PR has been closed, so the change to remove managed will need to be added to this PR

peteharverson · 2022-01-17T16:16:41Z

...transform/public/app/sections/transform_management/components/transform_list/use_actions.tsx

        {startAction.isModalVisible && <StartActionModal {...startAction} />}
+        {stopAction.isModalVisible && <StopActionModal {...stopAction} />}
+
        {editAction.config && editAction.isFlyoutVisible && (


We should also add the warning if the user tries to edit the transform.

Updated here a7f844b

peteharverson · 2022-01-18T17:04:44Z

x-pack/plugins/transform/public/app/common/transform.ts

+export const overrideTransformForCloning = (originalConfig: TransformConfigUnion) => {
+  // 'Managed' means job is preconfigured and deployed by other solutions
+  // we should not clone this setting
+  return { ...originalConfig, _meta: { ...originalConfig._meta, managed: false } };


Nit - should we be adding

"_meta": { "managed": false }

when cloning any transform (including ones which aren't managed), or not adding the managed field at all?

I think managed:false is more explicit but to match with what we have in ML, I removed it altogether here a13f502

The problem with setting it to false is that we'll only get transforms (or AD jobs) with "managed": false in them if they are cloned.
The user may wonder why "managed": false is only appearing for their cloned transforms and not for other other transforms they've created.

If we want to always have a managed property in transforms and AD jobs we'll have to add this to the wizards.

peteharverson

Added one comment about cloning transforms, but otherwise tested latest edits and LGTM.

jgowdyelastic

LGTM

qn895 · 2022-01-19T14:48:07Z

@elasticmachine merge upstream

kibana-ci · 2022-01-19T16:07:50Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`ml`	1592	1596	+4
`transform`	290	293	+3
total			+7

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`ml`	287	289	+2

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
`ml`	8	10	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`infra`	997.0KB	997.1KB	+90.0B
`ml`	3.5MB	3.5MB	+8.1KB
`transform`	360.3KB	363.3KB	+3.0KB
total			+11.2KB

Unknown metric groups

API count

id	before	after	diff
`ml`	291	293	+2

History

💚 Build #18104 succeeded a13f502
💚 Build #18030 succeeded a7f844b
💚 Build #17601 succeeded af963d1
💚 Build #17281 succeeded a5131b5
💚 Build #17038 succeeded 3e2a40b
💚 Build #16765 succeeded 4820a46

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @qn895

kibanamachine · 2022-01-19T16:08:34Z

💔 Backport failed

The pull request could not be backported due to the following error:
There are no branches to backport to. Aborting.

How to fix

Re-run the backport manually:

node scripts/backport --pr 122305

Questions ?

Please refer to the Backport tool documentation

## Summary This PR makes the following updates to the pre-built Security ML jobs: - Making the `security-packetbeat` compatible with Agent - Removing superfluous fields from the job configurations to make them consistent - Updating the `detector_description` field for almost all jobs - Adding influencers where missing and/or relevant - Adding a `job_revision` custom setting similar to the Logs [jobs](https://github.com/elastic/kibana/blob/main/x-pack/plugins/ml/server/models/data_recognizer/modules/logs_ui_analysis/ml/log_entry_rate.json#L29). Moving forward, this number will be updated each time a job is updated. We are starting with 4 since the `linux` and `windows` jobs are at v3 right now - Adding a `managed`: `true` tag to indicate that these jobs are pre-configured by Elastic and so users will see the warnings added in [this](#122305) PR if users choose to delete, or modify these jobs --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

qn895 added 5 commits December 28, 2021 11:18

Add managed badge

bb166b1

Add managed to module configs

a407aed

Fix message

ecde40d

Add management to managed transforms

9e620bb

Merge remote-tracking branch 'upstream/main' into ml-ad-managed-badges

a9ba547

qn895 self-assigned this Jan 5, 2022

qn895 added Feature:Anomaly Detection ML anomaly detection Feature:Transforms Transforms release_note:enhancement v8.1.0 :ml labels Jan 5, 2022

qn895 requested review from jgowdyelastic, lcawl and peteharverson January 5, 2022 18:38

lcawl reviewed Jan 5, 2022

View reviewed changes

...ns/ml/public/application/jobs/jobs_list/components/confirm_modals/edit_job_confirm_modal.tsx Outdated Show resolved Hide resolved

lcawl reviewed Jan 5, 2022

View reviewed changes

...ublic/application/jobs/jobs_list/components/confirm_modals/start_datafeeds_confirm_modal.tsx Outdated Show resolved Hide resolved

peteharverson reviewed Jan 6, 2022

View reviewed changes

qn895 marked this pull request as ready for review January 6, 2022 15:00

qn895 requested a review from a team as a code owner January 6, 2022 15:00

Update texts

427be5b

Handle bool in custom settings table

65cdb0a

lcawl reviewed Jan 6, 2022

View reviewed changes

...ublic/application/jobs/jobs_list/components/confirm_modals/start_datafeeds_confirm_modal.tsx Outdated Show resolved Hide resolved

Change message

cf88398

jgowdyelastic reviewed Jan 14, 2022

View reviewed changes

qn895 added 3 commits January 14, 2022 11:04

Merge remote-tracking branch 'upstream/main' into ml-ad-managed-badges

78ccf47

Add remove managed setting during clone in route

352559c

Add stopDatafeeds to declaration file

af963d1

peteharverson reviewed Jan 17, 2022

View reviewed changes

peteharverson self-requested a review January 17, 2022 16:17

qn895 added 2 commits January 18, 2022 08:29

Merge remote-tracking branch 'upstream/main' into ml-ad-managed-badges

4915fd7

Add call out when editing managed transform

a7f844b

qn895 requested a review from jgowdyelastic January 18, 2022 15:00

peteharverson reviewed Jan 18, 2022

View reviewed changes

peteharverson approved these changes Jan 18, 2022

View reviewed changes

Delete attr instead of setting it to false

a13f502

qn895 enabled auto-merge (squash) January 18, 2022 17:24

qn895 disabled auto-merge January 18, 2022 17:25

jgowdyelastic approved these changes Jan 19, 2022

View reviewed changes

Merge branch 'main' into ml-ad-managed-badges

93babe8

qn895 added the auto-backport Deprecated - use backport:version if exact versions are needed label Jan 19, 2022

qn895 enabled auto-merge (squash) January 19, 2022 14:48

qn895 merged commit 57ce437 into elastic:main Jan 19, 2022

kibanamachine added the backport:skip This PR does not require backporting label Jan 19, 2022

qn895 removed the auto-backport Deprecated - use backport:version if exact versions are needed label Jan 19, 2022

qn895 mentioned this pull request Jan 19, 2022

[ML] Add managed badge for anomaly detection jobs #120631

Closed

qn895 deleted the ml-ad-managed-badges branch February 9, 2022 16:32

ajosh0504 mentioned this pull request Apr 13, 2023

Updates to pre-built Security ML jobs #154596

Merged

Conversation

qn895 commented Jan 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Anomaly detection

Transforms

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Jan 6, 2022

Uh oh!

jgowdyelastic commented Jan 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgowdyelastic commented Jan 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgowdyelastic commented Jan 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgowdyelastic commented Jan 6, 2022

Uh oh!

jgowdyelastic commented Jan 6, 2022

Uh oh!

sophiec20 commented Jan 6, 2022

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qn895 Jan 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgowdyelastic Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peteharverson left a comment

Choose a reason for hiding this comment

Uh oh!

jgowdyelastic left a comment

Choose a reason for hiding this comment

Uh oh!

qn895 commented Jan 19, 2022

Uh oh!

kibana-ci commented Jan 19, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Any counts in public APIs

Async chunks

API count

History

Uh oh!

kibanamachine commented Jan 19, 2022

qn895 commented Jan 5, 2022 •

edited

Loading

jgowdyelastic commented Jan 6, 2022 •

edited

Loading

jgowdyelastic commented Jan 6, 2022 •

edited

Loading

jgowdyelastic commented Jan 6, 2022 •

edited

Loading

qn895 Jan 18, 2022 •

edited

Loading

jgowdyelastic Jan 19, 2022 •

edited

Loading