Prevent future convertToMultiNamespaceType migrations by gsoldevila · Pull Request #147369 · elastic/kibana

gsoldevila · 2022-12-12T16:29:30Z

elasticmachine · 2022-12-12T16:29:33Z

Pinging @elastic/kibana-core (Team:Core)

pgayvallet · 2022-12-13T08:18:18Z

packages/core/saved-objects/core-saved-objects-migration-server-internal/src/kibana_migrator.ts


+// ensure plugins don't try to convert SO namespaceTypes after 8.0.0
+// see https://github.com/elastic/kibana/issues/147344
+const ALLOWED_CONVERT_VERSION = '8.0.0';


I'll defer to @rudolf, but are we sure we don't want to allow type owners to migrate to shareable until some versions to come? This PR basically hard block any type from being converted as soon as merged. I want to make sure we're fine with the consequences.

if I understood correctly, this is a pre-requisite to only migrate an index if necessary.

FWIW I just saw #147198, but it is converting from "agnostic".

My 2cents are: as claimed in #147344, if we are not reindexing when there are no mapping updates (or, at least, no breaking changes), we cannot support any further version unless we change the current logic to identify that we need reindex anyway because a conversion must happen.

WDYT?

yeah if someone REALLY REALLY REALLY needs this we'll need to re-enable and detect this to ensure we reindex (which again will introduce downtime).

Re #147198 this is sort of a unique case because it's a new saved object type, there aren't any files saved objects in users clusters.

afharo

Should we update the docs accordingly?

Deprecate the option:

kibana/packages/core/saved-objects/core-saved-objects-server/src/migration.ts

Lines 77 to 80 in 4f4c160

    
             /** 
        
              * The version in which this object type is being converted to a multi-namespace type 
        
              */ 
        
             readonly convertToMultiNamespaceTypeVersion?: string;

kibana/packages/core/saved-objects/core-saved-objects-server/src/saved_objects_type.ts

Line 117 in 167797c

convertToMultiNamespaceTypeVersion?: string;

The documentation page Sharing saved objects

pgayvallet

I think this is what we want 😅 . LGTM

pgayvallet · 2022-12-15T09:41:48Z

...ore/saved-objects/core-saved-objects-migration-server-internal/src/core/document_migrator.ts

      );
    }
    if (convertToMultiNamespaceTypeVersion) {
+      // CHECKPOINT 1


NIT: Checkpoint!

pgayvallet · 2022-12-15T09:42:50Z

...ore/saved-objects/core-saved-objects-migration-server-internal/src/core/document_migrator.ts

        `Invalid convertToMultiNamespaceTypeVersion for type ${type}. Expected value to be a semver, but got '${convertToMultiNamespaceTypeVersion}'.`
      );
-    } else if (Semver.lt(convertToMultiNamespaceTypeVersion, minimumConvertVersion)) {
+    } else if (convertVersion && Semver.neq(convertToMultiNamespaceTypeVersion, convertVersion)) {


My gut feeling would have been to keep 8.0.0 as the minimum version and introduces a new maximumConvertVersion that we would set either to 8.0.0 or the current version like 8.6.0. But I'm probably overthinking this.

That could we useful if in the future we wish to unfreeze the functionality.
But at that point, we will probably have to rollback future changes in migration logic as well, otherwise we would have unexpected results.

…amespace-type-migrations

gsoldevila · 2022-12-21T11:17:25Z

docs/developer/advanced/sharing-saved-objects.asciidoc

 This guide describes the "Sharing saved objects" effort, and the breaking changes that plugin developers need to be aware of for the planned
 8.0 release of {kib}. It also describes how developers can take advantage of this feature.

+`From *8.7.0*, as a step towards Zero-Down-Time upgrades, plugins are no longer allowed to switch to shareable saved objects.`


kibana-ci · 2022-12-21T12:35:36Z

💚 Build Succeeded

Buildkite Build
Commit: 6fbdfdd

Metrics [docs]

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`osquery`	1	2	+1

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`fleet`	61	67	+6
`osquery`	109	115	+6
`securitySolution`	439	445	+6
total			+20

References to deprecated APIs

id	before	after	diff
`@kbn/core-saved-objects-migration-server-internal`	4	48	+44
`@kbn/core-test-helpers-so-type-serializer`	0	8	+8
`actions`	46	50	+4
`alerting`	93	95	+2
`canvas`	113	117	+4
`cases`	0	10	+10
`dashboard`	78	80	+2
`data`	57	59	+2
`dataViews`	38	40	+2
`encryptedSavedObjects`	2	6	+4
`graph`	56	58	+2
`lens`	16	18	+2
`lists`	72	74	+2
`maps`	25	27	+2
`savedObjectsTagging`	1	3	+2
`savedSearch`	6	8	+2
`securitySolution`	339	349	+10
`visualizations`	71	73	+2
total			+106

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`fleet`	70	76	+6
`osquery`	110	117	+7
`securitySolution`	515	521	+6
total			+21

History

💚 Build #94093 succeeded 2ce5baa

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2022-12-21T17:19:01Z

💔 All backports failed

Status	Branch	Result
❌	8.6	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 147369

Questions ?

Please refer to the Backport tool documentation

Prevent future convertToMultiNamespaceType migrations

2ce5baa

gsoldevila requested a review from a team as a code owner December 12, 2022 16:29

pgayvallet reviewed Dec 13, 2022

View reviewed changes

afharo mentioned this pull request Dec 14, 2022

[SO Migrations] Do not allow convertToMultiNamespaceTypeVersion > '8.0.0' #147510

Closed

3 tasks

afharo reviewed Dec 14, 2022

View reviewed changes

pgayvallet approved these changes Dec 15, 2022

View reviewed changes

gsoldevila added 2 commits December 21, 2022 12:14

Address PR comments

ec69b92

Merge branch 'main' into kbn-147344-prevent-future-convert-to-multi-n…

4c6a0f3

…amespace-type-migrations

gsoldevila commented Dec 21, 2022

View reviewed changes

gsoldevila requested review from afharo and rudolf December 21, 2022 11:17

gsoldevila added 2 commits December 21, 2022 12:36

Update docs

5d0c374

Update docs, take 2

6fbdfdd

rudolf approved these changes Dec 21, 2022

View reviewed changes

gsoldevila merged commit 57dad8f into elastic:main Dec 21, 2022

gsoldevila added backport:skip This PR does not require backporting and removed backport:prev-minor labels Dec 21, 2022

rudolf added the Epic:ScaleMigrations Scale upgrade migrations to millions of saved objects label Jun 2, 2023

	/**
	* The version in which this object type is being converted to a multi-namespace type
	*/
	readonly convertToMultiNamespaceTypeVersion?: string;

Conversation

gsoldevila commented Dec 12, 2022 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Dec 12, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

afharo left a comment

Choose a reason for hiding this comment

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsoldevila Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Dec 21, 2022

💚 Build Succeeded

Metrics [docs]

ESLint disabled in files

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

History

Uh oh!

kibanamachine commented Dec 21, 2022

💔 All backports failed

Manual backport

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

gsoldevila commented Dec 12, 2022 •

edited by kibanamachine

Loading

gsoldevila Dec 21, 2022 •

edited

Loading