Skip to content

Update mappings if/when new SO types are introduced#197061

Merged
gsoldevila merged 4 commits intoelastic:mainfrom
gsoldevila:kbn-new-so-types-dev
Oct 24, 2024
Merged

Update mappings if/when new SO types are introduced#197061
gsoldevila merged 4 commits intoelastic:mainfrom
gsoldevila:kbn-new-so-types-dev

Conversation

@gsoldevila
Copy link
Copy Markdown
Member

@gsoldevila gsoldevila commented Oct 21, 2024

@gsoldevila gsoldevila added Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor labels Oct 21, 2024
@gsoldevila gsoldevila requested a review from a team as a code owner October 21, 2024 13:28
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-core (Team:Core)

@gsoldevila gsoldevila added the bug Fixes for quality problems that affect the customer experience label Oct 21, 2024
@gsoldevila gsoldevila force-pushed the kbn-new-so-types-dev branch from 9448665 to b07a05c Compare October 21, 2024 15:58
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

  • 💔 Build #244374 failed 68924840d2b79629adca96a0871eed2a48e04a79
  • 💔 Build #244280 failed 94486659e197168677f849cec07f36c7ba2b44db

// compatible migration: ONLY new SO types have been introduced, skip directly to UPDATE_TARGET_MAPPINGS_META
return {
...stateP,
controlState: 'UPDATE_TARGET_MAPPINGS_META',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

your test makes it look like this is working, but I'm unsure when do we actually update the mappings properties since UPDATE_TARGET_MAPPINGS_META specifically doesn't update the properties https://github.com/elastic/kibana/blob/main/packages/core/saved-objects/core-saved-objects-migration-server-internal/src/next.ts#L240

Copy link
Copy Markdown
Member Author

@gsoldevila gsoldevila Oct 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mappings properties are already updated in the UPDATE_SOURCE_MAPPING_PROPERTIES step:

image

diffMappings will yield true with this PR whenever new types are introduced. This will in turn cause the mappings properties to be updated straight away in UPDATE_SOURCE_MAPPING_PROPERTIES, to check if the updates are compatible (aka if ES does not complain). See https://github.com/elastic/kibana/blob/main/packages/core/saved-objects/core-saved-objects-migration-server-internal/src/actions/update_source_mappings_properties.ts#L61

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For your peace of mind, I did test this with the integration test below:

expect(mappings.properties?.recent).toEqual({
  properties: {
    name: {
      type: 'keyword',
    },
  },
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I saw the test, so knew I must be missing something, thanks that makes sense!

@gsoldevila gsoldevila requested a review from rudolf October 23, 2024 12:52
@gsoldevila gsoldevila merged commit 8de3636 into elastic:main Oct 24, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.15, 8.16, 8.x

https://github.com/elastic/kibana/actions/runs/11495412111

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 24, 2024
## Summary

Addresses elastic/elastic-entity-model#70
Fixes regression introduced in
elastic#176803

(cherry picked from commit 8de3636)
@kibanamachine
Copy link
Copy Markdown
Contributor

💔 Some backports could not be created

Status Branch Result
8.15 Backport failed because of merge conflicts
8.16
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [Migrations] Provide testing archives + tooling for migrations integration tests (#193328)

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 197061

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 24, 2024
…197573)

# Backport

This will backport the following commits from `main` to `8.16`:
- [Update mappings if/when new SO types are introduced
(#197061)](#197061)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gerard
Soldevila","email":"gerard.soldevila@elastic.co"},"sourceCommit":{"committedDate":"2024-10-24T08:21:43Z","message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Core","release_note:skip","v9.0.0","backport:prev-minor","backport:prev-major","v8.16.0","v8.15.3"],"title":"Update
mappings if/when new SO types are
introduced","number":197061,"url":"https://github.com/elastic/kibana/pull/197061","mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},"sourceBranch":"main","suggestedTargetBranches":["8.16","8.15"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197061","number":197061,"mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},{"branch":"8.16","label":"v8.16.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.15","label":"v8.15.3","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Gerard Soldevila <gerard.soldevila@elastic.co>
@gsoldevila
Copy link
Copy Markdown
Member Author

💚 All backports created successfully

Status Branch Result
8.x
8.15

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

gsoldevila added a commit to gsoldevila/kibana that referenced this pull request Oct 24, 2024
## Summary

Addresses elastic/elastic-entity-model#70
Fixes regression introduced in
elastic#176803

(cherry picked from commit 8de3636)

# Conflicts:
#	src/core/server/integration_tests/saved_objects/migrations/group1/v2_migration.test.ts
#	src/core/server/integration_tests/saved_objects/migrations/group2/multiple_kb_nodes.test.ts
#	src/core/server/integration_tests/saved_objects/migrations/kibana_migrator_test_kit.fixtures.ts
gsoldevila added a commit to gsoldevila/kibana that referenced this pull request Oct 24, 2024
## Summary

Addresses elastic/elastic-entity-model#70
Fixes regression introduced in
elastic#176803

(cherry picked from commit 8de3636)

# Conflicts:
#	src/core/server/integration_tests/saved_objects/migrations/group1/v2_migration.test.ts
gsoldevila added a commit that referenced this pull request Oct 24, 2024
…197598)

# Backport

This will backport the following commits from `main` to `8.15`:
- [Update mappings if/when new SO types are introduced
(#197061)](#197061)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gerard
Soldevila","email":"gerard.soldevila@elastic.co"},"sourceCommit":{"committedDate":"2024-10-24T08:21:43Z","message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Core","release_note:skip","v9.0.0","backport:prev-minor","backport:prev-major","v8.16.0","v8.15.3"],"number":197061,"url":"https://github.com/elastic/kibana/pull/197061","mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197061","number":197061,"mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},{"branch":"8.16","label":"v8.16.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/197573","number":197573,"state":"MERGED","mergeCommit":{"sha":"3687435cb4531b1a0470abf42f69a420e2041cd2","message":"[8.16]
Update mappings if/when new SO types are introduced (#197061)
(#197573)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.16`:\n- [Update mappings if/when new SO types are
introduced\n(#197061)](https://github.com/elastic/kibana/pull/197061)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Gerard\nSoldevila\",\"email\":\"gerard.soldevila@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2024-10-24T08:21:43Z\",\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.17.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"Team:Core\",\"release_note:skip\",\"v9.0.0\",\"backport:prev-minor\",\"backport:prev-major\",\"v8.16.0\",\"v8.15.3\"],\"title\":\"Update\nmappings
if/when new SO types
are\nintroduced\",\"number\":197061,\"url\":\"https://github.com/elastic/kibana/pull/197061\",\"mergeCommit\":{\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.16\",\"8.15\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/197061\",\"number\":197061,\"mergeCommit\":{\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.15\",\"label\":\"v8.15.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Gerard Soldevila
<gerard.soldevila@elastic.co>"}},{"branch":"8.15","label":"v8.15.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
gsoldevila added a commit that referenced this pull request Oct 24, 2024
…197597)

# Backport

This will backport the following commits from `main` to `8.x`:
- [Update mappings if/when new SO types are introduced
(#197061)](#197061)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Gerard
Soldevila","email":"gerard.soldevila@elastic.co"},"sourceCommit":{"committedDate":"2024-10-24T08:21:43Z","message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Core","release_note:skip","v9.0.0","backport:prev-minor","backport:prev-major","v8.16.0","v8.15.3"],"number":197061,"url":"https://github.com/elastic/kibana/pull/197061","mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/197061","number":197061,"mergeCommit":{"message":"Update
mappings if/when new SO types are introduced (#197061)\n\n##
Summary\r\n\r\nAddresses
https://github.com/elastic/elastic-entity-model/issues/70\r\nFixes
regression introduced
in\r\nhttps://github.com//pull/176803","sha":"8de3636e43be7c874b2c3457f1496a0fc31f224d"}},{"branch":"8.16","label":"v8.16.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/197573","number":197573,"state":"MERGED","mergeCommit":{"sha":"3687435cb4531b1a0470abf42f69a420e2041cd2","message":"[8.16]
Update mappings if/when new SO types are introduced (#197061)
(#197573)\n\n# Backport\n\nThis will backport the following commits from
`main` to `8.16`:\n- [Update mappings if/when new SO types are
introduced\n(#197061)](https://github.com/elastic/kibana/pull/197061)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Gerard\nSoldevila\",\"email\":\"gerard.soldevila@elastic.co\"},\"sourceCommit\":{\"committedDate\":\"2024-10-24T08:21:43Z\",\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.17.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"Team:Core\",\"release_note:skip\",\"v9.0.0\",\"backport:prev-minor\",\"backport:prev-major\",\"v8.16.0\",\"v8.15.3\"],\"title\":\"Update\nmappings
if/when new SO types
are\nintroduced\",\"number\":197061,\"url\":\"https://github.com/elastic/kibana/pull/197061\",\"mergeCommit\":{\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.16\",\"8.15\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/197061\",\"number\":197061,\"mergeCommit\":{\"message\":\"Update\nmappings
if/when new SO types are introduced
(#197061)\\n\\n##\nSummary\\r\\n\\r\\nAddresses\nhttps://github.com/elastic/elastic-entity-model/issues/70\\r\\nFixes\nregression
introduced\nin\\r\\nhttps://github.com//pull/176803\",\"sha\":\"8de3636e43be7c874b2c3457f1496a0fc31f224d\"}},{\"branch\":\"8.16\",\"label\":\"v8.16.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"},{\"branch\":\"8.15\",\"label\":\"v8.15.3\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Gerard Soldevila
<gerard.soldevila@elastic.co>"}},{"branch":"8.15","label":"v8.15.3","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// v8.15.3 v8.15.4 v8.16.0 v8.17.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants