Skip to content

[SecuritySolution] Load entity store indices from security solution data view#195862

Merged
machadoum merged 9 commits intoelastic:mainfrom
machadoum:entity-store-enrich-processor-for-rebase
Oct 14, 2024
Merged

[SecuritySolution] Load entity store indices from security solution data view#195862
machadoum merged 9 commits intoelastic:mainfrom
machadoum:entity-store-enrich-processor-for-rebase

Conversation

@machadoum
Copy link
Copy Markdown
Member

@machadoum machadoum commented Oct 11, 2024

Summary

  • Update the Entity Store to retrieve indices from the security solution data view.
  • Create a new API that updates all installed entity engine indices (api/entity_store/engines/apply_dataview_indices)

How to test it?

  • Install the entity store
  • Check if the transform index has the security solutions data view indices
  • Call apply_dataview_indices API; it should not return changes
  • Update the security solution data view indices
  • Call apply_dataview_indices API and if the API response contains the updated indices
  • Check if the transform index also got updated

@machadoum machadoum force-pushed the entity-store-enrich-processor-for-rebase branch 3 times, most recently from e0df46d to b239b33 Compare October 11, 2024 15:24
@machadoum machadoum changed the title [SecuritySolution] [Merged with Mark's branch] Load entity store indices from security solution data view [SecuritySolution] Load entity store indices from security solution data view Oct 11, 2024
@machadoum machadoum force-pushed the entity-store-enrich-processor-for-rebase branch from b239b33 to de43d22 Compare October 11, 2024 15:53
namespace,
entityType,
fieldHistoryLength: 10, // we are not using this value so it can be anything
indexPatterns: [], // we are not using this value so it can be anything
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hop-dev Is fixing this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@machadoum machadoum self-assigned this Oct 11, 2024
@machadoum machadoum added backport:prev-minor Theme: entity_analytics Feature:Entity Analytics Security Solution Entity Analytics features Team:Entity Analytics Security Entity Analytics Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Oct 11, 2024
@machadoum machadoum marked this pull request as ready for review October 11, 2024 16:01
@machadoum machadoum requested review from a team as code owners October 11, 2024 16:01
@machadoum machadoum requested a review from hop-dev October 11, 2024 16:01
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

@machadoum machadoum enabled auto-merge (squash) October 14, 2024 10:54
Comment thread x-pack/plugins/entity_manager/server/lib/entity_client.ts Outdated
Comment thread x-pack/plugins/entity_manager/server/lib/entity_client.ts Outdated
Copy link
Copy Markdown
Member

@simianhacker simianhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some feedback around the EntityClient.

Copy link
Copy Markdown
Member

@simianhacker simianhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@machadoum machadoum merged commit 489c090 into elastic:main Oct 14, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11334897518

@elasticmachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #12 / Users filter should search on given search string on enter

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 20.8MB 20.8MB +11.0B

History

cc @machadoum

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 14, 2024
…ata view (elastic#195862)

## Summary

* Update the Entity Store to retrieve indices from the security solution
data view.
* Create a new API that updates all installed entity engine indices
(`api/entity_store/engines/apply_dataview_indices`)

### How to test it?
* Install the entity store
* Check if the transform index has the security solutions data view
indices
* Call `apply_dataview_indices` API; it should not return changes
* Update the security solution data view indices
* Call `apply_dataview_indices` API and if the API response contains the
updated indices
* Check if the transform index also got updated

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 489c090)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 15, 2024
…tion data view (#195862) (#196209)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[SecuritySolution] Load entity store indices from security solution
data view (#195862)](#195862)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Pablo
Machado","email":"pablo.nevesmachado@elastic.co"},"sourceCommit":{"committedDate":"2024-10-14T20:56:58Z","message":"[SecuritySolution]
Load entity store indices from security solution data view
(#195862)\n\n## Summary\r\n\r\n* Update the Entity Store to retrieve
indices from the security solution\r\ndata view.\r\n* Create a new API
that updates all installed entity engine
indices\r\n(`api/entity_store/engines/apply_dataview_indices`)\r\n\r\n\r\n###
How to test it?\r\n* Install the entity store\r\n* Check if the
transform index has the security solutions data view\r\nindices\r\n*
Call `apply_dataview_indices` API; it should not return changes\r\n*
Update the security solution data view indices\r\n* Call
`apply_dataview_indices` API and if the API response contains
the\r\nupdated indices\r\n* Check if the transform index also got
updated\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"489c0901ffd335879d9652424ab15ef9f39cc4cb","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","Team:
SecuritySolution","backport:prev-minor","Theme:
entity_analytics","Feature:Entity Analytics","Team:Entity
Analytics"],"title":"[SecuritySolution] Load entity store indices from
security solution data
view","number":195862,"url":"https://github.com/elastic/kibana/pull/195862","mergeCommit":{"message":"[SecuritySolution]
Load entity store indices from security solution data view
(#195862)\n\n## Summary\r\n\r\n* Update the Entity Store to retrieve
indices from the security solution\r\ndata view.\r\n* Create a new API
that updates all installed entity engine
indices\r\n(`api/entity_store/engines/apply_dataview_indices`)\r\n\r\n\r\n###
How to test it?\r\n* Install the entity store\r\n* Check if the
transform index has the security solutions data view\r\nindices\r\n*
Call `apply_dataview_indices` API; it should not return changes\r\n*
Update the security solution data view indices\r\n* Call
`apply_dataview_indices` API and if the API response contains
the\r\nupdated indices\r\n* Check if the transform index also got
updated\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"489c0901ffd335879d9652424ab15ef9f39cc4cb"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195862","number":195862,"mergeCommit":{"message":"[SecuritySolution]
Load entity store indices from security solution data view
(#195862)\n\n## Summary\r\n\r\n* Update the Entity Store to retrieve
indices from the security solution\r\ndata view.\r\n* Create a new API
that updates all installed entity engine
indices\r\n(`api/entity_store/engines/apply_dataview_indices`)\r\n\r\n\r\n###
How to test it?\r\n* Install the entity store\r\n* Check if the
transform index has the security solutions data view\r\nindices\r\n*
Call `apply_dataview_indices` API; it should not return changes\r\n*
Update the security solution data view indices\r\n* Call
`apply_dataview_indices` API and if the API response contains
the\r\nupdated indices\r\n* Check if the transform index also got
updated\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"489c0901ffd335879d9652424ab15ef9f39cc4cb"}}]}]
BACKPORT-->

Co-authored-by: Pablo Machado <pablo.nevesmachado@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature:Entity Analytics Security Solution Entity Analytics features release_note:enhancement Team:Entity Analytics Security Entity Analytics Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Theme: entity_analytics v8.16.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants