[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker#210585
Merged
lgestc merged 139 commits intoelastic:mainfrom Mar 31, 2025
Merged
[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker#210585lgestc merged 139 commits intoelastic:mainfrom
lgestc merged 139 commits intoelastic:mainfrom
Conversation
lgestc
changed the title
...tions/security/plugins/security_solution/public/timelines/store/middlewares/timeline_save.ts
Outdated
Show resolved
Hide resolved
Contributor
|
Lastly, there's a weird behavior where if I have a temporary dataView created in one timeline, then when I go to another one, I can't select it. I'm assuming it's because the temp dataview has no matches for it's index patterns, so maybe instead of creating an adhoc dataview when the index patterns don't match any existing indices...we just throw an error and show the default?.... 🤔 Screen.Recording.2025-03-25.at.5.03.28.PM.mov |
Contributor
Author
this should not be a problem, and while testing it I am able to select the adhoc one just fine... one problem is i am no longer able to import ndjson mentioned above, and has to "break" the existing timeline by removing a reference to the data view manually using kibana dev tools - maybe there is something more to it. we should get together and try to debug it during our meeting I think. |
elastic-vault-github-plugin-prod
bot
requested a review
from a team
as a code owner
| const DiscoverContainer = discover.DiscoverContainer; | ||
|
|
||
| const isLoading = Boolean(!dataView); | ||
| // TODO: (DV_PICKER) this should not work like that |
michaelolo24
approved these changes
Mar 26, 2025
Contributor
michaelolo24
left a comment
michaelolo24
left a comment
There was a problem hiding this comment.
Thank you for all of these changes!
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
Unknown metric groupsESLint disabled line counts
References to deprecated APIs
Total ESLint disabled count
History
|
cqliu1
pushed a commit
to cqliu1/kibana
that referenced
this pull request
Mar 31, 2025
…iew Picker (elastic#210585) # Unified Data View Picker: Phase 1 Implementation Part of elastic/security-team#11959 ## What This PR Does This PR represents the first step in our transition from the current Sourcerer component to the new unified Data View Picker. Specifically, this implementation: - Creates a new Data View Picker component - Implements feature flag protection for all changes - Handles asynchronous effects through Redux listener middleware - Establishes a new Redux store architecture to support ad hoc data views infrastructure - Utilizes ad hoc data views to handle legacy patterns from series 7 (replacing the previous upgrade data view flow) See the readme for more info: ```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md``` ## What This PR Does NOT Cover - Does not affect screens other than Timelines - Does not modify the existing Sourcerer component in any way - Does not fully support all URL/local storage patterns ## Implementation Notes We've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including: - Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases - There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete ## Testing Instructions 1. Add the following feature flag to your configuration: ``` xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled'] ``` 2. Navigate to the Timelines interface 3. Test interactions with the new Data View Picker --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
christineweng
pushed a commit
to christineweng/kibana
that referenced
this pull request
Jun 6, 2025
…iew Picker (elastic#210585) # Unified Data View Picker: Phase 1 Implementation Part of elastic/security-team#11959 ## What This PR Does This PR represents the first step in our transition from the current Sourcerer component to the new unified Data View Picker. Specifically, this implementation: - Creates a new Data View Picker component - Implements feature flag protection for all changes - Handles asynchronous effects through Redux listener middleware - Establishes a new Redux store architecture to support ad hoc data views infrastructure - Utilizes ad hoc data views to handle legacy patterns from series 7 (replacing the previous upgrade data view flow) See the readme for more info: ```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md``` ## What This PR Does NOT Cover - Does not affect screens other than Timelines - Does not modify the existing Sourcerer component in any way - Does not fully support all URL/local storage patterns ## Implementation Notes We've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including: - Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases - There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete ## Testing Instructions 1. Add the following feature flag to your configuration: ``` xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled'] ``` 2. Navigate to the Timelines interface 3. Test interactions with the new Data View Picker --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 9679f29) # Conflicts: # x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts # x-pack/solutions/security/plugins/security_solution/public/common/utils/timeline/use_show_timeline.test.tsx # x-pack/solutions/security/plugins/security_solution/public/common/utils/timeline/use_show_timeline_for_path.ts # x-pack/solutions/security/plugins/security_solution/public/timelines/components/modal/header/index.tsx # x-pack/solutions/security/plugins/security_solution/public/timelines/components/timeline/tabs/esql/index.tsx # x-pack/solutions/security/plugins/security_solution/public/timelines/store/middlewares/timeline_save.ts
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
christineweng
pushed a commit
to christineweng/kibana
that referenced
this pull request
Jun 6, 2025
…iew Picker (elastic#210585) # Unified Data View Picker: Phase 1 Implementation Part of elastic/security-team#11959 ## What This PR Does This PR represents the first step in our transition from the current Sourcerer component to the new unified Data View Picker. Specifically, this implementation: - Creates a new Data View Picker component - Implements feature flag protection for all changes - Handles asynchronous effects through Redux listener middleware - Establishes a new Redux store architecture to support ad hoc data views infrastructure - Utilizes ad hoc data views to handle legacy patterns from series 7 (replacing the previous upgrade data view flow) See the readme for more info: ```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md``` ## What This PR Does NOT Cover - Does not affect screens other than Timelines - Does not modify the existing Sourcerer component in any way - Does not fully support all URL/local storage patterns ## Implementation Notes We've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including: - Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases - There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete ## Testing Instructions 1. Add the following feature flag to your configuration: ``` xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled'] ``` 2. Navigate to the Timelines interface 3. Test interactions with the new Data View Picker --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 9679f29) # Conflicts: # x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts # x-pack/solutions/security/plugins/security_solution/public/common/utils/timeline/use_show_timeline.test.tsx # x-pack/solutions/security/plugins/security_solution/public/common/utils/timeline/use_show_timeline_for_path.ts # x-pack/solutions/security/plugins/security_solution/public/timelines/components/modal/header/index.tsx # x-pack/solutions/security/plugins/security_solution/public/timelines/components/timeline/tabs/esql/index.tsx # x-pack/solutions/security/plugins/security_solution/public/timelines/store/middlewares/timeline_save.ts
christineweng
added a commit
that referenced
this pull request
Jun 10, 2025
… Data View Picker (#210585) (#223044) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker (#210585)](#210585) - [[Security Solution] Rename use_data_view to use_data_view_spec #216461](#216461) - [[Security Solution] Rename use full data view hook #216614](#216614) - [[Security Solution] Replace sourcerer in global header #216685](#216685) - [[Security Solution] Remove .title use in use_selected_patterns #216994](#216994) - [[Security Solution] Render default security solution data view with managed label #216961](#216961) - [[Security Solution] Replace sourcerer in analyzer #218183](#218183) - [[Security Solution] Replace use_sourcerer_data_view #216997](#216997) - [[Security Solution] Replace sourcerer in EQL tab with dataview picker #218897](#218897) - [[Security Solution][Sourcerer] replace use get scoped data view #220196](#220196) - [[Security Solution] renaming dataView to dataViewSpec and adding types for clarity #220718](#220718) - [[Security Solution][Sourcerer] Maintain url sync support #221737](#221737) - [[Security Solution][Data View Manager] Allow passing data view to query bar #220585](#220585) - [[Security Solution] Fix data view picker privilege #222122](#222122) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Luke Gmys","email":"11671118+lgestc@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-03-31T12:12:57Z","message":"[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker (#210585)\n\n# Unified Data View Picker: Phase 1 Implementation\nPart of https://github.com/elastic/security-team/issues/11959\n\n## What This PR Does\nThis PR represents the first step in our transition from the current\nSourcerer component to the new unified Data View Picker. Specifically,\nthis implementation:\n- Creates a new Data View Picker component\n- Implements feature flag protection for all changes\n- Handles asynchronous effects through Redux listener middleware\n- Establishes a new Redux store architecture to support ad hoc data\nviews infrastructure\n- Utilizes ad hoc data views to handle legacy patterns from series 7\n(replacing the previous upgrade data view flow)\n\nSee the readme for more info: \n```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md```\n\n## What This PR Does NOT Cover\n- Does not affect screens other than Timelines\n- Does not modify the existing Sourcerer component in any way\n- Does not fully support all URL/local storage patterns\n\n## Implementation Notes\nWe've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including:\n- Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases\n- There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete\n\n## Testing Instructions\n1. Add the following feature flag to your configuration:\n ```\n xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled']\n ```\n2. Navigate to the Timelines interface\n3. Test interactions with the new Data View Picker\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"9679f2941550856d75e00c1faadd8c9669afe917","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:skip","Team: SecuritySolution","Team:Threat Hunting:Investigations","Feature:Sourcerer","9.1 candidate","v9.1.0"],"title":"[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker","number":210585,"url":"https://github.com/elastic/kibana/pull/210585","mergeCommit":{"message":"[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker (#210585)\n\n# Unified Data View Picker: Phase 1 Implementation\nPart of https://github.com/elastic/security-team/issues/11959\n\n## What This PR Does\nThis PR represents the first step in our transition from the current\nSourcerer component to the new unified Data View Picker. Specifically,\nthis implementation:\n- Creates a new Data View Picker component\n- Implements feature flag protection for all changes\n- Handles asynchronous effects through Redux listener middleware\n- Establishes a new Redux store architecture to support ad hoc data\nviews infrastructure\n- Utilizes ad hoc data views to handle legacy patterns from series 7\n(replacing the previous upgrade data view flow)\n\nSee the readme for more info: \n```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md```\n\n## What This PR Does NOT Cover\n- Does not affect screens other than Timelines\n- Does not modify the existing Sourcerer component in any way\n- Does not fully support all URL/local storage patterns\n\n## Implementation Notes\nWe've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including:\n- Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases\n- There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete\n\n## Testing Instructions\n1. Add the following feature flag to your configuration:\n ```\n xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled']\n ```\n2. Navigate to the Timelines interface\n3. Test interactions with the new Data View Picker\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"9679f2941550856d75e00c1faadd8c9669afe917"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210585","number":210585,"mergeCommit":{"message":"[Security Solution][Sourcerer] Replace Sourcerer with Discover Data View Picker (#210585)\n\n# Unified Data View Picker: Phase 1 Implementation\nPart of https://github.com/elastic/security-team/issues/11959\n\n## What This PR Does\nThis PR represents the first step in our transition from the current\nSourcerer component to the new unified Data View Picker. Specifically,\nthis implementation:\n- Creates a new Data View Picker component\n- Implements feature flag protection for all changes\n- Handles asynchronous effects through Redux listener middleware\n- Establishes a new Redux store architecture to support ad hoc data\nviews infrastructure\n- Utilizes ad hoc data views to handle legacy patterns from series 7\n(replacing the previous upgrade data view flow)\n\nSee the readme for more info: \n```x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.md```\n\n## What This PR Does NOT Cover\n- Does not affect screens other than Timelines\n- Does not modify the existing Sourcerer component in any way\n- Does not fully support all URL/local storage patterns\n\n## Implementation Notes\nWe've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including:\n- Some interfaces might look odd, especially the hooks that return the data view or patterns - this is intentional to support existing use cases\n- There are feature flag-based conditional statements throughout the code that will be removed once the transition is complete\n\n## Testing Instructions\n1. Add the following feature flag to your configuration:\n ```\n xpack.securitySolution.enableExperimental: ['newDataViewPickerEnabled']\n ```\n2. Navigate to the Timelines interface\n3. Test interactions with the new Data View Picker\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"9679f2941550856d75e00c1faadd8c9669afe917"}}]}] BACKPORT--> --------- Co-authored-by: Luke Gmys <11671118+lgestc@users.noreply.github.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unified Data View Picker: Phase 1 Implementation
Part of https://github.com/elastic/security-team/issues/11959
What This PR Does
This PR represents the first step in our transition from the current Sourcerer component to the new unified Data View Picker. Specifically, this implementation:
See the readme for more info:
x-pack/solutions/security/plugins/security_solution/public/data_view_manager/readme.mdWhat This PR Does NOT Cover
Implementation Notes
We've made several accommodations to support both Sourcerer and the new Data View Picker simultaneously during this transition period, including:
Testing Instructions