[ML] Clone analytics job #59791

darnautov · 2020-03-10T16:53:48Z

Summary

Adds support of analytics job cloning.

In case the job has been created using the UI form the flyout for cloning appears with pre-populated inputs except for the job id and destination index. Besides, the clone action detects any settings that are not supported by the form and opens the advanced editor straightaway.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
This was checked for cross-browser compatibility, including a check against IE11

elasticmachine · 2020-03-10T16:53:51Z

Pinging @elastic/ml-ui (:ml)

darnautov · 2020-03-10T17:00:03Z

There is an issue with the context menu and not only visual. All the other context items are initialized with Eui button components which end up with nested buttons in the result DOM, hence you can observe some react warnings. I'm going to fix it in this PR.

peteharverson · 2020-03-10T18:07:49Z

Gave this a quick test, and I think we also need to handle cloning of an analytics job which was created using the advanced editor. For example, I switched to the advanced editor and added one of the optional params documented here.

When I then clone that job, the flyout opens in the standard 'form' mode and my advanced parameter is lost. We should probably check to see if the job config contains fields or values which aren't editable in the form editor, and if so, switch the flyout to the advanced mode.

alvarezmelissa87 · 2020-03-10T19:37:50Z

Great addition overall! 😄 Thanks for your work on this. Just have a couple of questions/comments. Everything works as expected, aside from the issue below.

Even though the model_memory_limit from the source job is part of the data that gets cloned over the call to _explain (that usually fetches that estimate) is still happening.

This can cause an override of the cloned value if the origin job had a source index that is not an index pattern (created via dev tools, for example)

You can see the source_index doesn't get populated in the form (though it does when advanced editor is open) though the dependent_variable does.

Would it be worth ensuring that extra _explain call doesn't happen initially? Or just handling origin jobs that may have been created with an index that has not had an index pattern created 🤔

...on/data_frame_analytics/pages/analytics_management/components/analytics_list/action_clone.ts

peteharverson · 2020-03-11T17:28:10Z

I am having a few issues where the cloned model memory limit is cleared and set back to the default, which is below the minimum required, so the attempt to start the job fails after saving (I have been using the bank classification job from the ML QA dev bootstrap df_analytics suite).

…s-jobs

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts

…lytics_button.tsx

…-clone-analytics-jobs

peteharverson

Tested and LGTM. Just some minor comments on the code.

peteharverson · 2020-03-12T17:18:09Z

...ta_frame_analytics/pages/analytics_management/components/analytics_list/action_clone.test.ts

+        source: {
+          index: ['glass_withoutdupl_norm'],
+          query: {
+            // TODO check default for `match`


Did you mean to leave in this comment?

I noticed one of the outlier detection jobs (created by env bootstrap) has query match instead of match_all. As I saw in all the other cases match_all is a default, so probably it's just a typo in a bootstrap job.

...n/data_frame_analytics/pages/analytics_management/components/analytics_list/action_clone.tsx

...ta_frame_analytics/pages/analytics_management/components/analytics_list/action_clone.test.ts

alvarezmelissa87 · 2020-03-12T20:22:13Z

I'm having an issue creating new analytics jobs - when I click 'create' on the form after filling it in I get a blank screen with this in the console.

I'm only seeing this when it's the very first analytics job being created.

Also, the Excludes dropdown is not getting populated initially. Apologies for missing this in the slack convo about the explain endpoint being called twice. It would still need to be called once just to populate the excludes dropdown.

…n the fist update

darnautov · 2020-03-13T09:43:49Z

Thanks @alvarezmelissa87 , good spot with excluded options and missing input ref! Just pushed the fixes.

peteharverson

Tested latest edits, and LGTM. I didn't test the issue you found @alvarezmelissa87 when creating your first analytics job - would you be able to confirm that is fixed for you please.

alvarezmelissa87

Latest edits LGTM ⚡️

…s-jobs # Conflicts: # x-pack/plugins/ml/public/application/data_frame_analytics/pages/analytics_management/components/create_analytics_form/create_analytics_form.tsx # x-pack/test/functional/services/machine_learning/api.ts

kibanamachine · 2020-03-15T21:37:02Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 526f88c

History

💔 Build #33303 failed bfd985b
💔 Build #33258 failed a9b742a
💛 Build #33252 was flaky fde5626
💔 Build #33129 failed cac0b36
💚 Build #33012 succeeded 5939ddc

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* [ML] clone analytics job * [ML] flyout clone header * [ML] improve clone action context menu item * [ML] support advanced job cloning * [ML] extractCloningConfig * [ML] fix isAdvancedSetting condition, add test * [ML] clone job header * [ML] job description placeholder * [ML] setEstimatedModelMemoryLimit on source index change * [ML] Fix types. * [ML] useUpdateEffect in create_analytics_form.tsx * [ML] setJobClone action * [ML] remove CreateAnalyticsFlyoutWrapper instance from the create_analytics_button.tsx * [ML] fix types * [ML] hack to align Clone button with the other actions * [ML] unknown props lead to advanced editor * [ML] rename maximum_number_trees ot max_trees * [ML] fix forceInput * [ML] populate excludesOptions on the first update, skip setting mml on the fist update * [ML] init functional test for cloning analytics jobs * [ML] functional tests * [ML] fix functional tests imports * [ML] fix indices names for functional tests * [ML] functional tests for outlier detection and regression jobs cloning * [ML] delete james tag * [ML] fix tests arrangement Co-authored-by: Walter Rafelsberger <[email protected]>

* [ML] clone analytics job * [ML] flyout clone header * [ML] improve clone action context menu item * [ML] support advanced job cloning * [ML] extractCloningConfig * [ML] fix isAdvancedSetting condition, add test * [ML] clone job header * [ML] job description placeholder * [ML] setEstimatedModelMemoryLimit on source index change * [ML] Fix types. * [ML] useUpdateEffect in create_analytics_form.tsx * [ML] setJobClone action * [ML] remove CreateAnalyticsFlyoutWrapper instance from the create_analytics_button.tsx * [ML] fix types * [ML] hack to align Clone button with the other actions * [ML] unknown props lead to advanced editor * [ML] rename maximum_number_trees ot max_trees * [ML] fix forceInput * [ML] populate excludesOptions on the first update, skip setting mml on the fist update * [ML] init functional test for cloning analytics jobs * [ML] functional tests * [ML] fix functional tests imports * [ML] fix indices names for functional tests * [ML] functional tests for outlier detection and regression jobs cloning * [ML] delete james tag * [ML] fix tests arrangement Co-authored-by: Walter Rafelsberger <[email protected]> Co-authored-by: Walter Rafelsberger <[email protected]>

…o alerting/view-in-app * 'alerting/view-in-app' of github.com:gmmorris/kibana: (33 commits) [SIEM] Adds 'Closes and opens signals' Cypress test (elastic#59950) [NP] Graph migration (elastic#59409) [ML] Clone analytics job (elastic#59791) Move VALUE_CLICK_TRIGGER and APPLY_FILTER_TRIGGER to ui_action… (elastic#60202) Handle improperly defined Watcher Logging Action text parameter. (elastic#60169) Move select range trigger to uiActions (elastic#60168) [SIEM][CASES] Configure cases: Final (elastic#59358) Closes elastic#59784. Sets xpack.apm.serviceMapEnabled default value true. (elastic#60153) [siem/cypress] update junit filename to be picked up by runbld (elastic#60156) OSS logic added to test environment (elastic#60134) Move canvas setup into app mount (elastic#59926) enable triggers_actions_ui plugin by default (elastic#60137) Expose Elasticsearch from start and deprecate from setup (elastic#59886) [SIEM] [Case] Insert timeline bugfix and limit 25 cases (elastic#60136) [ML] Client side cut over (elastic#60100) Disable query bar on service map routes (elastic#60118) Move subscribe_with_scope to kibana_legacy (elastic#59781) [Ingest] Agent Config create with sys monitoring (elastic#60111) [Watcher UI] Not possible to edit a watch that was created with the API if the ID contains a dot (elastic#59383) Actually fetch functionbeat fields needed for telemetry (elastic#60054) ...

* master: (40 commits) skips 'config_open.ts' files from linter check (elastic#60248) [Searchprofiler] Spacing between rendered shards (elastic#60238) Add UiSettings validation & Kibana default route redirection (elastic#59694) [SIEM][CASE] Change configuration button (elastic#60229) [Step 1][App Arch] Saved object migrations from kibana plugin to new platform (elastic#59044) adds new test (elastic#60064) [Uptime] Index Status API to Rest (elastic#59657) [SIEM] Adds 'Closes and opens signals' Cypress test (elastic#59950) [NP] Graph migration (elastic#59409) [ML] Clone analytics job (elastic#59791) Move VALUE_CLICK_TRIGGER and APPLY_FILTER_TRIGGER to ui_action… (elastic#60202) Handle improperly defined Watcher Logging Action text parameter. (elastic#60169) Move select range trigger to uiActions (elastic#60168) [SIEM][CASES] Configure cases: Final (elastic#59358) Closes elastic#59784. Sets xpack.apm.serviceMapEnabled default value true. (elastic#60153) [siem/cypress] update junit filename to be picked up by runbld (elastic#60156) OSS logic added to test environment (elastic#60134) Move canvas setup into app mount (elastic#59926) enable triggers_actions_ui plugin by default (elastic#60137) Expose Elasticsearch from start and deprecate from setup (elastic#59886) ...

darnautov added 3 commits March 10, 2020 16:47

[ML] clone analytics job

ef9170c

[ML] flyout clone header

86be82e

[ML] improve clone action context menu item

4923423

darnautov added release_note:enhancement :ml v8.0.0 Feature:Data Frame Analytics ML data frame analytics features v7.7.0 labels Mar 10, 2020

darnautov requested review from walterra, alvarezmelissa87 and peteharverson March 10, 2020 16:53

darnautov requested a review from a team as a code owner March 10, 2020 16:53

darnautov self-assigned this Mar 10, 2020

darnautov added 2 commits March 11, 2020 16:05

[ML] support advanced job cloning

307b32b

[ML] extractCloningConfig

d1356bc

peteharverson reviewed Mar 11, 2020

View reviewed changes

...on/data_frame_analytics/pages/analytics_management/components/analytics_list/action_clone.ts Outdated Show resolved Hide resolved

darnautov and others added 6 commits March 11, 2020 21:30

[ML] fix isAdvancedSetting condition, add test

ba2cabd

[ML] clone job header

9fc4fdb

Merge remote-tracking branch 'upstream/master' into ML-clone-analytic…

a667557

…s-jobs

[ML] job description placeholder

84deee6

[ML] setEstimatedModelMemoryLimit on source index change

6722186

[ML] Fix types.

e476f28

walterra reviewed Mar 12, 2020

View reviewed changes

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts Outdated Show resolved Hide resolved

darnautov added 2 commits March 12, 2020 13:04

[ML] useUpdateEffect in create_analytics_form.tsx

ff14676

[ML] setJobClone action

254daf4

darnautov added 5 commits March 12, 2020 14:42

[ML] remove CreateAnalyticsFlyoutWrapper instance from the create_ana…

2c180c1

…lytics_button.tsx

Merge remote-tracking branch 'origin/ML-clone-analytics-jobs' into ML…

29ca176

…-clone-analytics-jobs

[ML] fix types

9f05bb8

[ML] hack to align Clone button with the other actions

458ec21

[ML] unknown props lead to advanced editor

d3992a0

peteharverson approved these changes Mar 12, 2020

View reviewed changes

darnautov added 3 commits March 13, 2020 09:42

[ML] rename maximum_number_trees ot max_trees

44cc41b

[ML] fix forceInput

6b7d0bf

[ML] populate excludesOptions on the first update, skip setting mml o…

5939ddc

…n the fist update

peteharverson approved these changes Mar 13, 2020

View reviewed changes

alvarezmelissa87 approved these changes Mar 13, 2020

View reviewed changes

darnautov added 3 commits March 13, 2020 18:51

[ML] init functional test for cloning analytics jobs

cac0b36

[ML] functional tests

a9b742a

darnautov requested a review from pheyos March 14, 2020 13:36

darnautov added 5 commits March 15, 2020 11:46

[ML] fix functional tests imports

fee64e2

[ML] fix indices names for functional tests

8c2968e

[ML] functional tests for outlier detection and regression jobs cloning

bfd985b

[ML] delete james tag

24d4b57

[ML] fix tests arrangement

526f88c

darnautov merged commit 96ac1aa into elastic:master Mar 15, 2020

darnautov deleted the ML-clone-analytics-jobs branch March 15, 2020 21:41

darnautov mentioned this pull request Mar 16, 2020

[7.x] [ML] Clone analytics job (#59791) #60228

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Clone analytics job #59791

[ML] Clone analytics job #59791

darnautov commented Mar 10, 2020 •

edited

Loading

elasticmachine commented Mar 10, 2020

darnautov commented Mar 10, 2020

peteharverson commented Mar 10, 2020 •

edited

Loading

alvarezmelissa87 commented Mar 10, 2020

peteharverson commented Mar 11, 2020

peteharverson left a comment

peteharverson Mar 12, 2020

darnautov Mar 13, 2020

alvarezmelissa87 commented Mar 12, 2020 •

edited

Loading

darnautov commented Mar 13, 2020

peteharverson left a comment

alvarezmelissa87 left a comment

kibanamachine commented Mar 15, 2020

[ML] Clone analytics job #59791

[ML] Clone analytics job #59791

Conversation

darnautov commented Mar 10, 2020 • edited Loading

Summary

Adds support of analytics job cloning.

Checklist

elasticmachine commented Mar 10, 2020

darnautov commented Mar 10, 2020

peteharverson commented Mar 10, 2020 • edited Loading

alvarezmelissa87 commented Mar 10, 2020

peteharverson commented Mar 11, 2020

peteharverson left a comment

Choose a reason for hiding this comment

peteharverson Mar 12, 2020

Choose a reason for hiding this comment

darnautov Mar 13, 2020

Choose a reason for hiding this comment

alvarezmelissa87 commented Mar 12, 2020 • edited Loading

darnautov commented Mar 13, 2020

peteharverson left a comment

Choose a reason for hiding this comment

alvarezmelissa87 left a comment

Choose a reason for hiding this comment

kibanamachine commented Mar 15, 2020

💚 Build Succeeded

History

darnautov commented Mar 10, 2020 •

edited

Loading

peteharverson commented Mar 10, 2020 •

edited

Loading

alvarezmelissa87 commented Mar 12, 2020 •

edited

Loading