Skip to content

Deprecate universal entity#210978

Merged
CohenIdo merged 8 commits intoelastic:mainfrom
CohenIdo:deprecate-universal-entity
Feb 18, 2025
Merged

Deprecate universal entity#210978
CohenIdo merged 8 commits intoelastic:mainfrom
CohenIdo:deprecate-universal-entity

Conversation

@CohenIdo
Copy link
Copy Markdown
Contributor

@CohenIdo CohenIdo commented Feb 13, 2025

Summary

Following our decision to change the architecture from extracting entities from logs and events that contains multiple entities in one entry, and focus on documents that represent only one entity (more on that here and here),
this PR deprecates all the logic behind universal entity:

  • Disable universal entity initialization when the entity store is enabled.
  • Universal Entity Store Definition
  • keyword builder ingest pipeline
  • Metadata extractor
  • Tests
  • Feature flag

@CohenIdo CohenIdo requested a review from tiansivive February 13, 2025 10:55
@CohenIdo CohenIdo added release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related labels Feb 13, 2025
@CohenIdo CohenIdo marked this pull request as ready for review February 13, 2025 12:45
@CohenIdo CohenIdo requested review from a team as code owners February 13, 2025 12:45
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@CohenIdo CohenIdo added the backport:skip This PR does not require backporting label Feb 13, 2025
@hop-dev
Copy link
Copy Markdown
Contributor

hop-dev commented Feb 13, 2025

@CohenIdo universal entity is in the 8.18, 8.x (8.19) and 9.0.0 branches too, so I think we need the labels backport:version, v9.0.0 , v8.19.0 and v8.18.0 on this PR. Does that sound right?

@CohenIdo
Copy link
Copy Markdown
Contributor Author

CohenIdo commented Feb 13, 2025

@CohenIdo universal entity is in the 8.18, 8.x (8.19) and 9.0.0 branches too, so I think we need the labels backport:version, v9.0.0 , v8.19.0 and v8.18.0 on this PR. Does that sound right?

@hop-dev I agree, but I’m not sure it’s crucial since it’s behind a feature flag and nothing is really affected, wdyt?

@hop-dev
Copy link
Copy Markdown
Contributor

hop-dev commented Feb 13, 2025

@CohenIdo if we have the opportunity to remove it we should go for it, it might make backporting easier in the future, the more we can keep them in sync the easier our life is!

@CohenIdo
Copy link
Copy Markdown
Contributor Author

@CohenIdo if we have the opportunity to remove it we should go for it, it might make backporting easier in the future, the more we can keep them in sync the easier our life is!

I can agree with that:) I added the labels.

@CohenIdo CohenIdo added v9.0.0 backport:version Backport to applied version labels v8.18.0 v8.19.0 labels Feb 13, 2025
@CohenIdo CohenIdo force-pushed the deprecate-universal-entity branch from 5cd9e0d to 597041c Compare February 13, 2025 16:56
@CohenIdo CohenIdo requested a review from a team as a code owner February 13, 2025 16:56
@CohenIdo CohenIdo force-pushed the deprecate-universal-entity branch from 5d306e7 to 5580257 Compare February 17, 2025 11:49
Copy link
Copy Markdown
Contributor

@tiansivive tiansivive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also all good with me 🚀

@CohenIdo CohenIdo force-pushed the deprecate-universal-entity branch from 1c2e9dc to fb38624 Compare February 18, 2025 08:35
Copy link
Copy Markdown
Contributor

@albertoblaz albertoblaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM though I'm not very knowledgable about this part of the implementation

Copy link
Copy Markdown
Contributor

@JordanSh JordanSh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove FE side changes form this PR.
While removing universal as an entity is one thing, the Universal Flyout is another thing and will still exist in order to display other entities than user, host, and service

Changes required for the Universal Flyout are handled here and should not concern the removal of this entity.

@elastic elastic deleted a comment from JordanSh Feb 18, 2025
@CohenIdo CohenIdo force-pushed the deprecate-universal-entity branch from f5e9ef9 to f65b8c4 Compare February 18, 2025 12:10
Copy link
Copy Markdown
Contributor

@JordanSh JordanSh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 8.9MB 8.9MB -397.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 85.2KB 85.2KB -30.0B

History

  • 💔 Build #277137 failed f5e9ef917312cf730a06c9043ca1add04af68f74
  • 💔 Build #276883 failed 1c2e9dc0618eb953817284b70a5fbeef7ebdd9cc
  • 💔 Build #276074 failed 5d306e7f4ee49e88e0a95fc215841592f2dab62d
  • 💔 Build #275946 failed 5cd9e0d462d2c393edff8afaf43fbae26da24b53

@CohenIdo CohenIdo merged commit f5c9529 into elastic:main Feb 18, 2025
@CohenIdo CohenIdo removed the backport:skip This PR does not require backporting label Feb 19, 2025
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.18, 8.x, 9.0

https://github.com/elastic/kibana/actions/runs/13412673840

@kibanamachine
Copy link
Copy Markdown
Contributor

💔 All backports failed

Status Branch Result
8.18 Backport failed because of merge conflicts
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [Security Solution][Endpoint] Add FTR API tests that validates creation of DOT indices (#197899)
9.0 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 210978

Questions ?

Please refer to the Backport tool documentation

@CohenIdo
Copy link
Copy Markdown
Contributor Author

💚 All backports created successfully

Status Branch Result
9.0
8.x
8.18

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

CohenIdo added a commit that referenced this pull request Feb 20, 2025
# Backport

This will backport the following commits from `main` to `8.x`:
- [Deprecate universal
entity](#210978)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Ido
Cohen","email":"90558359+CohenIdo@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-02-18T14:49:32Z","message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Cloud
Security","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"Deprecate
universal
entity","number":210978,"url":"https://github.com/elastic/kibana/pull/210978","mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210978","number":210978,"mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CohenIdo added a commit that referenced this pull request Feb 20, 2025
# Backport

This will backport the following commits from `main` to `8.18`:
- [Deprecate universal
entity](#210978)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Ido
Cohen","email":"90558359+CohenIdo@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-02-18T14:49:32Z","message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Cloud
Security","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"Deprecate
universal
entity","number":210978,"url":"https://github.com/elastic/kibana/pull/210978","mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210978","number":210978,"mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CohenIdo added a commit that referenced this pull request Feb 20, 2025
# Backport

This will backport the following commits from `main` to `9.0`:
- [Deprecate universal
entity](#210978)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Ido
Cohen","email":"90558359+CohenIdo@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-02-18T14:49:32Z","message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Cloud
Security","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"Deprecate
universal
entity","number":210978,"url":"https://github.com/elastic/kibana/pull/210978","mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210978","number":210978,"mergeCommit":{"message":"Deprecate
universal
entity","sha":"f5c9529e37ea3dddca4db748bfa7e2391303f87e"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related v8.18.0 v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants