[8.6][ML Inference] Verify pipeline usage before deletion by demjened · Pull Request #144053 · elastic/kibana

demjened · 2022-10-26T17:20:36Z

Summary

This PR adds a check to the DELETE /internal/enterprise_search/indices/{indexName}/ml_inference/pipeline_processors/{pipelineName} endpoint. Before deleting the supplied ML inference processor pipeline from the given index, we check whether the same pipeline is referenced in a different index (or more precisely, in the {other_index_name}@ml-inference managed pipeline).

Manual testing

ml-inference-ner2 is referenced in two indices' managed pipelines:

Deleting the ml-inference-ner3 pipeline succeeds

Deleting the ml-inference-ner2 pipeline fails

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

…-fix'

…jened/kibana into create-ml-inference-pipeline

…-fix'

…-ref HEAD~1..HEAD --fix'

…-fix'

…-ref HEAD~1..HEAD --fix'

…-fix'

…-ref HEAD~1..HEAD --fix'

…-fix'

...erver/lib/indices/pipelines/ml_inference/pipeline_processors/delete_ml_inference_pipeline.ts

brianmcgue

A few comments/questions, but overall, LGTM

x-pack/plugins/enterprise_search/server/routes/enterprise_search/indices.ts

x-pack/plugins/enterprise_search/server/routes/enterprise_search/indices.test.ts

...erver/lib/indices/pipelines/ml_inference/pipeline_processors/delete_ml_inference_pipeline.ts

…-ref HEAD~1..HEAD --fix'

kibana-ci · 2022-10-28T19:11:52Z

💚 Build Succeeded

Buildkite Build
Commit: 92cc803

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`enterpriseSearch`	1.7MB	1.7MB	+42.0B

History

💔 Build #83612 failed 58436000024fd987305034ffe7d0d7c21c0826fb
💔 Build #83569 failed 9a9247a9254414a8a80a4d449a69d843e06ab12a
💔 Build #83336 failed 94afd27682519225661e634c4dfad2e582dd23a1
💔 Build #83329 failed f52568e79e945f354342573ad325975a97055590
💚 Build #82931 succeeded 66d007aba669d5cceb02a724ad0631eb0776cc7b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

.../lib/indices/pipelines/ml_inference/pipeline_processors/delete_ml_inference_pipeline.test.ts

* main: (41 commits) [api-docs] Daily api_docs build (elastic#144212) Add readonly view to role management (elastic#143893) [api-docs] Daily api_docs build (elastic#144208) [APM] Adds button group to navigate to "All services" (elastic#142911) Update react-query to ^4.12.0 (main) (elastic#139986) [APM] Support specific fields when creating service groups (elastic#142201) (elastic#143881) [api-docs] Daily api_docs build (elastic#144203) [ts] add stub index.d.ts in @kbn/ui-shared-deps-npm [Synthetics] Fix failing Synthetics Integration test (elastic#144175) chore(NA): remove @types/pkg link creation when generating a new package (elastic#144200) [Osquery] Update schema to v5.5.1 (elastic#144090) [ci] remove github-checks-reporter (elastic#144193) [8.6][ML Inference] Verify pipeline usage before deletion (elastic#144053) [ts] ts refs cache was removed, remove capture task Added Rollups CCS Test (elastic#144074) [auto] migrate existing plugin/package configs [ts] stop building @types packages in bootstrap skip failing test suite (elastic#142762) skip failing test suite (elastic#144186) [Fleet] Show Add Fleet Server instead of add agent when adding agent from agent policy (elastic#144105) ...

demjened and others added 30 commits September 13, 2022 13:47

Add ML inference PL creation flow

c9a0545

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

e293860

…-fix'

Add exists check, clean up code a bit

8fb6d63

Merge branch 'create-ml-inference-pipeline' of https://github.com/dem…

05f5eb2

…jened/kibana into create-ml-inference-pipeline

Fix dest name

d69478b

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

2443d8c

…-fix'

Separate concerns

49aebdb

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

aee672f

…-ref HEAD~1..HEAD --fix'

Remove i18n due to linter error, fix src field ref

d838052

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

f4d53ba

…-ref HEAD~1..HEAD --fix'

Add/update unit tests

ef14bad

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

5d3b37d

…-ref HEAD~1..HEAD --fix'

Refactor error handling

8613f5b

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

b972688

…-fix'

Add sub-pipeline to parent ML PL

92b4ec5

Add unit tests and docs

71bf27b

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

7c74509

…-ref HEAD~1..HEAD --fix'

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

0ecad7e

…-fix'

Merge branch 'main' into create-ml-inference-pipeline

ce8f0a7

Refactor error handling

1c15559

Wrap logic into higher level function

bf29f2a

Add route test

aaae199

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

9d2f232

…-ref HEAD~1..HEAD --fix'

Merge branch 'main' into create-ml-inference-pipeline

99a7027

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

63c52cb

…-fix'

Merge branch 'create-ml-inference-pipeline'

85a5020

Merge branch 'main' of https://github.com/elastic/kibana

1f70e59

Merge branch 'main' of https://github.com/elastic/kibana

58f6524

Merge branch 'main' of https://github.com/elastic/kibana

27028f8

Merge branch 'main' of https://github.com/elastic/kibana

8acd6a3

demjened added 2 commits October 25, 2022 15:14

Merge branch 'main' of https://github.com/elastic/kibana

eb61b85

Merge branch 'main' of https://github.com/elastic/kibana

56a002b

demjened added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:EnterpriseSearch v8.6.0 labels Oct 26, 2022

demjened commented Oct 26, 2022

View reviewed changes

...erver/lib/indices/pipelines/ml_inference/pipeline_processors/delete_ml_inference_pipeline.ts Outdated Show resolved Hide resolved

demjened requested review from TattdCodeMonkey and brianmcgue October 26, 2022 18:34

demjened marked this pull request as ready for review October 26, 2022 18:34

demjened requested a review from a team October 26, 2022 18:34

brianmcgue reviewed Oct 26, 2022

View reviewed changes

sphilipse reviewed Oct 27, 2022

View reviewed changes

demjened force-pushed the demjened/validate-pipeline-usage branch 2 times, most recently from 9a9247a to fbb10bf Compare October 28, 2022 16:20

demjened and others added 4 commits October 28, 2022 12:20

Merge branch 'main' of https://github.com/elastic/kibana

0da6f09

Add validation of pipeline usage before deletion

2e7a2e0

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

bef6419

…-ref HEAD~1..HEAD --fix'

Address CR

ac40591

demjened force-pushed the demjened/validate-pipeline-usage branch from fbb10bf to 5843600 Compare October 28, 2022 16:20

Use rich error object when pipeline is is use

e86622a

demjened force-pushed the demjened/validate-pipeline-usage branch from 5843600 to e86622a Compare October 28, 2022 17:53

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

92cc803

…-ref HEAD~1..HEAD --fix'

demjened requested review from brianmcgue and sphilipse October 28, 2022 18:58

brianmcgue approved these changes Oct 28, 2022

View reviewed changes

.../lib/indices/pipelines/ml_inference/pipeline_processors/delete_ml_inference_pipeline.test.ts Show resolved Hide resolved

demjened merged commit 7671176 into elastic:main Oct 28, 2022

demjened deleted the demjened/validate-pipeline-usage branch October 28, 2022 20:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[8.6][ML Inference] Verify pipeline usage before deletion#144053

[8.6][ML Inference] Verify pipeline usage before deletion#144053
demjened merged 52 commits intoelastic:mainfrom
demjened:demjened/validate-pipeline-usage

demjened commented Oct 26, 2022 •

edited

Loading

Uh oh!

Uh oh!

brianmcgue left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kibana-ci commented Oct 28, 2022

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

demjened commented Oct 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

Uh oh!

brianmcgue left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kibana-ci commented Oct 28, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

demjened commented Oct 26, 2022 •

edited

Loading