[Security Solution][Sourcerer] Default data view init flow for the DV manager by lgestc · Pull Request #221338 · elastic/kibana

lgestc · 2025-05-23T08:23:31Z

Summary

This PR reuses sourcerer data view creation logic for security solution and applies it to the new flow with data view manager (discover data view picker).

Closes https://github.com/elastic/security-team/issues/12570
Closes #220589

Testing

Starting kibana from scratch with the feature flag enabled should create default security solution view, in either default or custom Kibana Space.

The FF:

xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled']

Checklist

Check the PR satisfies following conditions.

Unit or functional tests were updated / added

elasticmachine · 2025-05-27T08:36:40Z

Pinging @elastic/security-threat-hunting-investigations (Team:Threat Hunting:Investigations)

...urity/plugins/security_solution/public/data_view_manager/hooks/use_init_data_view_manager.ts

christineweng · 2025-05-28T23:27:52Z

...ecurity/plugins/security_solution/public/data_view_manager/utils/create_default_data_view.ts

+    }
+
+    // check for/generate default Security Solution Kibana data view
+    const sourcererDataViews = await createSourcererDataView({


Is my understanding correct?

we want to retain the createSourererDataView method to get the default data views (plus renaming sourcerer reference in clean up)

we don't need the kibana data views since they are replaced in the shared redux slice

don't need signal either

not sure about signal

We do need signal for now, but won't ideally once the actual Security Solution Alerts dataView is created that only looks at that, then we shouldn't have to append it here and the detections scope can just be set to that one. But the same space specific logic will need to be applied to that one

Also, while not super impactful from a performance perspective, createSourcererDataView has a number of unnecessary loops over a pattern list and seemingly duplicated behavior around setting the dataview title property. Doesn't need to be part of this PR, but we can include cleaning that up as part of the cleanup for this work

...ty/plugins/security_solution/public/data_view_manager/utils/create_default_data_view.test.ts

...utions/security/plugins/security_solution/public/sourcerer/containers/use_signal_helpers.tsx

x-pack/solutions/security/plugins/security_solution/public/common/store/store.ts

michaelolo24 · 2025-06-03T02:48:13Z

~~It looks like the dataview loading error is still happening in the newly created spaces~~

Ignore this, my UI was just in a weird state.

One thing, can you update the references to this hardcoded pattern here for the default space security-solution-default:

kibana/x-pack/solutions/security/plugins/security_solution/public/data_view_manager/constants.ts

Line 8 in d0fd87e

    
           export const DEFAULT_SECURITY_SOLUTION_DATA_VIEW_ID = 'security-solution-default';

? You can probably just either reference the id from the store (preferred) or create a hook that utilized useSpaceId

...ecurity/plugins/security_solution/public/data_view_manager/utils/create_default_data_view.ts

lgestc · 2025-06-18T14:34:46Z

...utions/security/plugins/security_solution/public/sourcerer/containers/use_init_sourcerer.tsx

+  /* eslint-disable react-hooks/rules-of-hooks */
+  // NOTE: skipping the entire hook on purpose when the new picker is enabled
+  // will be removed as part of the cleanup in https://github.com/elastic/security-team/issues/11959
+  if (newDataViewPickerEnabled) {


this will prevent the sourcerer init actions entirely if the new picker is enabled

lgestc

added a few comments for the reviewers

elasticmachine · 2025-06-18T15:27:29Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 3571299

Failed CI Steps

FTR Configs #114

Test Failures

[job] [logs] FTR Configs #114 / alerting api integration security and spaces enabled - Group 2 Connectors ServiceNow ITSM ServiceNow ITSM - Executor Execution closeIncident should close the incident

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	7635	7637	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	9.4MB	9.4MB	+2.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	94.8KB	94.8KB	+6.0B

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`securitySolution`	94	95	+1

ESLint disabled line counts

id	before	after	diff
`securitySolution`	620	621	+1

Total ESLint disabled count

id	before	after	diff
`securitySolution`	714	716	+2

History

💚 Build #309865 succeeded 0a34eeb
💛 Build #309219 was flaky 0a6ceca
💔 Build #308792 failed 7abb46b

christineweng

LGTM, great work @lgestc 👏

michaelolo24 · 2025-06-19T02:09:04Z

...ity/plugins/security_solution/public/data_view_manager/redux/listeners/data_view_selected.ts

 import type { DataViewManagerScopeName } from '../../constants';

+/**
+ * Creates a Redux listener for handling data view selection logic in the data view manager.


...security/plugins/security_solution/public/data_view_manager/redux/listeners/init_listener.ts

michaelolo24 · 2025-06-19T02:11:38Z

...security/plugins/security_solution/public/data_view_manager/redux/listeners/init_listener.ts

+        // NOTE: save default data id for the given space in the store.
+        // this is used to identify the default selection in pickers across Kibana Space
+        listenerApi.dispatch(
+          sharedDataViewManagerSlice.actions.setDefaultDataViewId(defaultDataView.id)


michaelolo24 · 2025-06-19T02:13:09Z

x-pack/solutions/security/plugins/security_solution/public/data_view_manager/redux/types.ts

+
+export interface SignalIndexMetadata {
+  name: string;
+  isOutdated: boolean;


Can you add a comment for this?

michaelolo24 · 2025-06-19T02:19:50Z

.../security_solution/public/detections/containers/detection_engine/alerts/use_signal_index.tsx

  });
+  const experimentalSignalIndexMappingOutdated = useSelector(signalIndexOutdatedSelector);
+
+  const signalIndexMappingOutdated = newDataViewPickerEnabled


👍🏾 , I would just point to this for the isOutdated line in the store

michaelolo24

Nice work! Thanks for fixing the spaces dataView creation. Works well now 👍🏾

…a_view_manager/redux/listeners/init_listener.ts Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>

kibanamachine · 2025-06-19T09:25:41Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15754430337

… manager (elastic#221338) ## Summary This PR reuses sourcerer data view creation logic for security solution and applies it to the new flow with data view manager (discover data view picker). Closes elastic/security-team#12570 Closes elastic#220589 ## Testing Starting kibana from scratch with the feature flag enabled should create default security solution view, **in either default or custom Kibana Space**. The FF: ``` xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled'] ``` ### Checklist Check the PR satisfies following conditions. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated / added --------- Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co> (cherry picked from commit e6ed157)

kibanamachine · 2025-06-19T09:31:56Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

… the DV manager (#221338) (#224534) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution][Sourcerer] Default data view init flow for the DV manager (#221338)](#221338)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Luke Gmys <11671118+lgestc@users.noreply.github.com> Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>

lgestc added 5 commits May 23, 2025 10:21

--wip-- [skip ci]

88c4be6

--wip-- [skip ci]

12582f8

--wip-- [skip ci]

a9147a1

--wip-- [skip ci]

95c4093

--wip-- [skip ci]

bd866c0

lgestc added 9.1 candidate Feature:Sourcerer backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes v9.1.0 Team:Threat Hunting:Investigations Security Solution Threat Hunting Investigations Team labels May 26, 2025

lgestc added 3 commits May 26, 2025 12:18

--wip-- [skip ci]

2f1f5d0

move signal init to manager hook

3e032ac

add missing index selector

2a572eb

elastic deleted a comment from elasticmachine May 27, 2025

naming

0519fea

lgestc marked this pull request as ready for review May 27, 2025 08:36

lgestc requested review from a team as code owners May 27, 2025 08:36

lgestc and others added 4 commits May 27, 2025 10:42

naming

37ff8fc

add tests for create default data view

03155b1

fix broken test

17a478c

Merge branch 'main' into default_data_view_init

29a7b5d

lgestc mentioned this pull request May 27, 2025

[Security Solution][Data View Manager] Support non-default space #220589

Closed

christineweng reviewed May 29, 2025

View reviewed changes

lgestc added 2 commits May 29, 2025 15:36

Merge branch 'main' into default_data_view_init

5487e61

Merge branch 'main' into default_data_view_init

40f17f4

michaelolo24 reviewed Jun 3, 2025

View reviewed changes

...ecurity/plugins/security_solution/public/data_view_manager/utils/create_default_data_view.ts Show resolved Hide resolved

lgestc added 4 commits June 18, 2025 12:24

re-enable use init sourcerer

f0832b7

skip sourcerer init entirely if the new picker is enabled

6ca3ca6

remove todo

b6f8814

use data view id from manager in poll index

0a34eeb

lgestc requested a review from christineweng June 18, 2025 13:28

Merge branch 'main' into default_data_view_init

3571299

lgestc added backport:prev-major and removed backport:skip This PR does not require backporting labels Jun 18, 2025

lgestc commented Jun 18, 2025

View reviewed changes

lgestc added backport:version Backport to applied version labels v8.19.0 and removed backport:prev-major labels Jun 18, 2025

christineweng approved these changes Jun 18, 2025

View reviewed changes

michaelolo24 reviewed Jun 19, 2025

View reviewed changes

...security/plugins/security_solution/public/data_view_manager/redux/listeners/init_listener.ts Outdated Show resolved Hide resolved

michaelolo24 reviewed Jun 19, 2025

View reviewed changes

michaelolo24 approved these changes Jun 19, 2025

View reviewed changes

lgestc and others added 2 commits June 19, 2025 09:32

Update x-pack/solutions/security/plugins/security_solution/public/dat…

08cdfde

…a_view_manager/redux/listeners/init_listener.ts Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>

Merge branch 'main' into default_data_view_init

fdbdee6

lgestc enabled auto-merge (squash) June 19, 2025 07:33

lgestc merged commit e6ed157 into elastic:main Jun 19, 2025
10 checks passed

kibanamachine mentioned this pull request Jun 19, 2025

[8.19] [Security Solution][Sourcerer] Default data view init flow for the DV manager (#221338) #224534

Merged

Conversation

lgestc commented May 23, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Checklist

Uh oh!

elasticmachine commented May 27, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

michaelolo24 commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lgestc left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 18, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Async chunks

Page load bundle

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

christineweng left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelolo24 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Jun 19, 2025

Uh oh!

kibanamachine commented Jun 19, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lgestc commented May 23, 2025 •

edited by kibanamachine

Loading

michaelolo24 commented Jun 3, 2025 •

edited

Loading