APEX-72 Enforce Kibana >= 8.18.0; Remove switchToModelVersionAt

[gsoldevila]

Summary

• Address Force Kibana upgrades to 8.18 / 8.19 before 9.x #217145 - Put in place a check to ensure we're upgrading from Kibana 8.18.0 or newer.
• Address APEX-72 Improve v2 logic to account for "older" migrationVersions in the index mappings #220521 - New attempt at removing the switchToModelVersionAt property, inspired on Removed switchToModelVersionAt from SO API definition #219029.

The previous attempt caused a regression: index meta information started storing modelVersions that were older than the previously stored ones, which were defaulting to 10.0.0 for SO types that did NOT define modelVersions.

This was due to the removal of the applyTypeDefaults, which was ensuring all SOs had the switchToModelVersionAt property set.

This flag was then used by src/core/packages/saved-objects/base-server-internal/src/model_version/version_map.ts to determine whether to use modelVersions or the legacy migrations property in order to determine the latest model version for a given type.

When removing the switchToModelVersionAt flag (and its default backfill), the logic started defaulting to the latest migrations version for those SO types that were not defining any modelVersion, resulting in older versions that those stored in the SO indices. This caused incident https://elasticco.atlassian.net/browse/INC-3818.

This regression has been shipped in 9.0.0 (the PR was backported), so in top of the cleanup, we now need to address #220521 to ensure a smooth transition OnPrem => Serverless.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[elasticmachine]

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

[rudolf]

haven't made it through the whole PR, but thought I'd leave this comment so long

[rudolf]

should we maybe do a runtime check to prevent modelversion 0? Otherwise you can define your type with just mappings and then at a later point define a transform or backfill modelVersion with 0 and I don't think that transform will be applied because the virtualVersionMap with useModelVersionsOnly=false stays 10.0.0.

I think a similar example might be bar with only migrations, if you add a modelVersion 0 then the virtual version still stays 10.0.0 so I think we'd miss mappings_addition.

[rudolf]

Essentially I don't think this test will pass:

      expect(
        getLatestMappingsModelVersion(
          buildType({
            migrations: {
              '7.17.2': dummyMigration,
              '8.6.0': dummyMigration,
            }
          })
        )
      ).not.toEqual(
        getLatestMappingsModelVersion(
          buildType({
            migrations: {
              '7.17.2': dummyMigration,
              '8.6.0': dummyMigration,
            },
            modelVersions: {
              0: dummyModelVersionWithMappingsChanges(),
            },
          })
        )
      );

[gsoldevila]

I used it as a hack for this test, but fair point, I can try to limit the possible values for the keys.

[jesuswr]

8.10.0 instead of 8.11.0?

[gsoldevila]

Yeah, weird that it was like that 🤷🏼 I'll update it.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: ea9a457

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/core-saved-objects-base-server-internal	183	182	-1
@kbn/core-saved-objects-migration-server-internal	94	96	+2
@kbn/core-saved-objects-server	137	136	-1
total			-0

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
@kbn/core	886	887	+1

Unknown metric groups

API count

id	before	after	diff
@kbn/core-saved-objects-migration-server-internal	127	130	+3
@kbn/core-saved-objects-server	574	572	-2
total			+1

References to deprecated APIs

id	before	after	diff
@kbn/core	638	633	-5

History

• 💔 Build #310818 failed 386a784
• 💔 Build #309451 failed aaf6256
• 💚 Build #306642 succeeded ec879ca
• 💔 Build #306565 failed 103cf2f
• 💔 Build #306562 failed 8ffb263

cc @gsoldevila