[ML] Switching to config files for serverless ML features by jgowdyelastic · Pull Request #166477 · elastic/kibana

jgowdyelastic · 2023-09-14T15:25:59Z

Switches to using the serverless config file to enabled/disable ML features rather than a function shared from the setup contract.
Storing these flags in a config file means they are already available when setup runs and so can be used when registering integrations into other plugins.

Removes the dependency on ML from security_solution_serverless, serverless_observability and serverless_search

…eatures

darnautov · 2023-09-20T09:15:32Z

x-pack/plugins/ml/server/lib/alerts/register_ml_alerts.ts

-  registerAnomalyDetectionAlertType(alertParams);
-  registerJobsMonitoringRuleType(alertParams);
+export function registerMlAlerts(alertParams: RegisterAlertParams, enabledFeatures: MlFeatures) {
+  if (enabledFeatures.ad === true) {


question/suggestion: did you consider performing a check before registerMlAlerts and registerCasesPersistableState are called?

I'm passing in the enabled features into registerMlAlerts and have the check happen in there.
I did it this way to allow for future non AD alerts to still be registered by this function. As in, the registerMlAlerts function will decide which alerts to register.

To move the check out into the plugin.ts file, these register functions would need to be renamed to have AD or similar in the name e.g. registerMlADAlerts
I thought it would be neater to have one function to register all ML alerts, and have it decide which ones to register based on feature enablement.

darnautov · 2023-09-20T09:16:25Z

x-pack/plugins/ml/server/plugin.ts


    initMlServerLog({ log: this.log });

    if (plugins.alerting) {


Suggested change

if (plugins.alerting) {

if (plugins.alerting && this.enabledFeatures.ad) {

darnautov · 2023-09-20T09:16:38Z

x-pack/plugins/ml/server/plugin.ts

+          mlSharedServices: sharedServicesProviders,
+          mlServicesProviders: internalServicesProviders,
+        },
+        this.enabledFeatures


Suggested change

this.enabledFeatures

darnautov · 2023-09-20T09:17:04Z

x-pack/plugins/ml/server/plugin.ts

@@ -323,8 +308,12 @@ export class MlServerPlugin
      }

      if (mlLicense.isMlEnabled() && mlLicense.isFullLicense()) {


darnautov

LGTM

semd

Security changes LGTM!

…eatures

sebelga

LGTM 👍

peteharverson · 2023-09-20T16:55:28Z

config/serverless.security.yml

 xpack.securitySolution.enableExperimental:
  - discoverInTimeline
+
+xpack.ml.ad.enabled: true


Probably not related to the changes here, but should the Memory usage page be accessible in the Security project? There is a link to it from the ML home page, but I get the 'Access denied' page.

The memory usage page is currently disabled in serverless.
This may be a mistake. I've pushed a fix to another WIP PR to change this. As that PR has some similar client side changes.

jloleysens

re-approving on behalf of core?

…eatures

peteharverson

Tested each project, and stateful, and LGTM. Just left a comment about renaming a file.

x-pack/plugins/ml/server/lib/register_sameple_data_set_links.ts

…eatures

kibana-ci · 2023-09-21T12:50:24Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: f3fdd7b
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-166477-f3fdd7bc3aeb

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #31 / serverless common UI Index Management Index Templates "before all" hook for "renders the index templates tab"

Metrics [docs]

Canvas Sharable Runtime

The Canvas "shareable runtime" is an bundle produced to enable running Canvas workpads outside of Kibana. This bundle is included in third-party webpages that embed canvas and therefor should be as slim as possible.

id	before	after	diff
`module count`	-	5614	+5614
`total size`	-	6.0MB	+6.0MB

History

💛 Build #160721 was flaky 2ebb3a1
💛 Build #160464 was flaky 9b76af2
💛 Build #160381 was flaky 04fa7ed
💛 Build #160036 was flaky 3eb3fab
💛 Build #159813 was flaky 9853c46

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jgowdyelastic

jgowdyelastic added 9 commits September 14, 2023 16:18

[ML] Switching to config files for serverless ML features

3a852bd

removing old code

4a10eb0

updating registering integrations

67d4864

correcting schema

1da14b2

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

68e2a89

…eatures

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

071f773

…eatures

fixing registrations

c0b6ef3

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

1071942

…eatures

fixing typo

d095a4b

jgowdyelastic requested review from darnautov and peteharverson September 19, 2023 06:58

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

157e63b

…eatures

jgowdyelastic marked this pull request as ready for review September 19, 2023 06:59

jgowdyelastic requested review from a team as code owners September 19, 2023 06:59

jgowdyelastic requested a review from a team September 19, 2023 06:59

jgowdyelastic requested review from a team as code owners September 19, 2023 06:59

jgowdyelastic requested review from a team September 19, 2023 06:59

jgowdyelastic requested review from a team as code owners September 19, 2023 06:59

jgowdyelastic requested a review from a team September 19, 2023 06:59

jgowdyelastic requested a review from a team as a code owner September 19, 2023 06:59

jgowdyelastic added 2 commits September 19, 2023 08:11

types

fd7fbe2

type

9853c46

jgowdyelastic requested review from darnautov and qn895 and removed request for darnautov September 19, 2023 08:16

registering routes based on enabled features

3eb3fab

jgowdyelastic requested a review from qn895 September 19, 2023 16:16

darnautov reviewed Sep 20, 2023

View reviewed changes

darnautov approved these changes Sep 20, 2023

View reviewed changes

semd approved these changes Sep 20, 2023

View reviewed changes

sphilipse approved these changes Sep 20, 2023

View reviewed changes

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

04fa7ed

…eatures

jgowdyelastic requested review from qn895 and removed request for qn895 September 20, 2023 15:47

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

9b76af2

…eatures

sebelga approved these changes Sep 20, 2023

View reviewed changes

peteharverson reviewed Sep 20, 2023

View reviewed changes

jgowdyelastic marked this pull request as draft September 21, 2023 07:20

jgowdyelastic marked this pull request as ready for review September 21, 2023 07:20

jloleysens approved these changes Sep 21, 2023

View reviewed changes

sphilipse approved these changes Sep 21, 2023

View reviewed changes

semd approved these changes Sep 21, 2023

View reviewed changes

sebelga self-requested a review September 21, 2023 09:29

sebelga approved these changes Sep 21, 2023

View reviewed changes

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

3aa1c1d

…eatures

jgowdyelastic added the ci:build-serverless-image label Sep 21, 2023

gbamparop approved these changes Sep 21, 2023

View reviewed changes

peteharverson approved these changes Sep 21, 2023

View reviewed changes

x-pack/plugins/ml/server/lib/register_sameple_data_set_links.ts Show resolved Hide resolved

jgowdyelastic added 2 commits September 21, 2023 11:52

fixing typo in file

2ebb3a1

Merge branch 'main' into switching-to-config-file-for-serverless-ml-f…

f3fdd7b

…eatures

jgowdyelastic merged commit e0ae59f into elastic:main Sep 21, 2023

kibanamachine added the backport:skip This PR does not require backporting label Sep 21, 2023

watson added the Project:Serverless Work as part of the Serverless project for its initial release label Oct 4, 2023

	if (plugins.alerting) {
	if (plugins.alerting && this.enabledFeatures.ad) {

		@@ -323,8 +308,12 @@ export class MlServerPlugin
		}

		if (mlLicense.isMlEnabled() && mlLicense.isFullLicense()) {

Conversation

jgowdyelastic commented Sep 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darnautov left a comment

Choose a reason for hiding this comment

Uh oh!

semd left a comment

Choose a reason for hiding this comment

Uh oh!

sebelga left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens left a comment

Choose a reason for hiding this comment

Uh oh!

peteharverson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibana-ci commented Sep 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Canvas Sharable Runtime

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

jgowdyelastic commented Sep 14, 2023 •

edited

Loading

kibana-ci commented Sep 21, 2023 •

edited

Loading