[Security Solution] remove newDataViewPickerEnabled feature flag

[PhilippeOberti]

Warning

This PR changes a lot of files. This change will not make it within 9.3.
9.3.0 feature freeze is on December 16th 2025, so ideally we'd be merging this PR on December 17th. This should give plenty of time to all the teams involved to do their testing.

Summary

The newDataViewPickerEnabled feature flag has been turned on in this previous PR. We have not had any issues related to the usage of the data view picker over the old sourcerer code.

This PR removes the newDataViewPickerEnabled as well as all the code related to it and to the old sourcerer implementation. Some hooks are still being used, so a follow up PR will finalize the cleanup.

What's included in this PR

Here's a list of high level changes:

• remove all sourcerer code related to the newDataViewPickerEnabled feature flag (some follow up cleanup will be required to fully delete the content of the sourcerer folder
• rename all the experimentalDataView (and other experimental... names) with just dataView

What work will remains for follow up PRs

What will be coming after this PR is merged:

• some more cleanup like rename all the remaining sourcererScope to pageScope
• remove some of the duplicated useDataView and other hooks that are still living throughout Security Solution
• move all the necessary hooks and functions left in the sourcerer folder into the new data_view_manager folder
• finally, move the code that lives in the data_view_manager folder into a package and update all imports accordingly

Caution

While the PR should only removed unused code, please take your time looking at the code and testing the pages and feature you own, to make sure that no unwanted code was removed and functionality impacted!

How to test

• pull down the branch
• smoke test the application

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

[vitaliidm]

Detection Engine area LGTM

[hop-dev]

Tested Entity Analytics screens all working fine 👍

[animehart]

Contextual Security area (K8S Dashboard) LGTM, tested locally

[NicholasPeretti]

Great effort! 🚀

Thank you for this clean-up! It's really nice to see so much code disappear ☺️

I've left a few comments with some NITs but no blockers — up to you if you want address them ☺️

[NicholasPeretti]

Great effort! 🚀

Thank you for this clean-up! It's really nice to see so much code disappear ☺️

I've left a few comments with some NITs but no blockers — up to you if you want address them ☺️

Ah, I just realised that my VSCode github integration didn't save the comments... great... 😡
Well don't worry, it was nothing big anyways 😄

[andrew-goldstein]

Thanks @PhilippeOberti for removing this code throughout the solution!
Local desk testing of the files owned by elastic/security-generative-ai looked good, apart from one error here, though the changes to that file itself look OK.
LGTM 🚀

[maximpn]

@PhilippeOberti It's nice to see a PR removing a lot of lines 😄

I've tested the PR locally to make sure there are no issues. The testing went good.

[macroscopeapp]

🟢 Low risk_score_management/risk_score_preview_section.tsx:172

RiskEnginePreview destructures only dataView from useDataView(PageScope.alerts) and ignores the status return value. When the dataView is still loading, dataView contains INITIAL_DV with a falsy title, so useRiskScorePreview receives an undefined data_view_id, disables its query, and isLoading becomes false. The component then renders the normal UI with empty data and no loading indicator, misleading users into thinking there are no results while the dataView is still being fetched. Consider checking status === 'ready' before rendering content that depends on dataView, similar to the pattern used in GraphVisualization.

Also found in 1 other location(s)

x-pack/solutions/security/plugins/security_solution/public/timelines/components/timeline/tabs/query/events_count.tsx:97

The call to dataView.getRuntimeMappings() on line 97 does not use optional chaining, unlike other dataView accesses in this component (e.g., line 94 uses dataView?.id). Before useDataView has loaded (when status is 'pristine' or 'loading'), dataView is the initial placeholder value INITIAL_DV. If this placeholder doesn't properly implement getRuntimeMappings(), the component will crash on initial render. The previous code used experimentalDataView?.getRuntimeMappings() ?? {} with both optional chaining and a fallback, which protected against this.

🤖 Copy this AI Prompt to have your agent fix this:

In file x-pack/solutions/security/plugins/security_solution/public/entity_analytics/components/risk_score_management/risk_score_preview_section.tsx around line 172:

`RiskEnginePreview` destructures only `dataView` from `useDataView(PageScope.alerts)` and ignores the `status` return value. When the dataView is still loading, `dataView` contains `INITIAL_DV` with a falsy `title`, so `useRiskScorePreview` receives an undefined `data_view_id`, disables its query, and `isLoading` becomes `false`. The component then renders the normal UI with empty data and no loading indicator, misleading users into thinking there are no results while the dataView is still being fetched. Consider checking `status === 'ready'` before rendering content that depends on `dataView`, similar to the pattern used in `GraphVisualization`.

Evidence trail:
x-pack/solutions/security/plugins/security_solution/public/entity_analytics/components/risk_score_management/risk_score_preview_section.tsx:172 - `const { dataView } = useDataView(PageScope.alerts);` shows only dataView destructured, status ignored

x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_data_view.ts:17-20 - `INITIAL_DV = new DataView({ fieldFormats: {} })` has no title

x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_data_view.ts:42 - `useState<DataView>(INITIAL_DV)` initializes with empty DataView

x-pack/solutions/security/plugins/security_solution/public/data_view_manager/hooks/use_data_view.ts:72-73 - hook returns `{ dataView, status }`

x-pack/solutions/security/plugins/security_solution/public/entity_analytics/api/hooks/use_preview_risk_scores.ts:67 - `{ enabled: !!dataViewId }` disables query when dataViewId is falsy

x-pack/solutions/security/plugins/security_solution/public/entity_analytics/components/risk_score_management/risk_score_preview_section.tsx:239 - `isLoading={isLoading}` passed without dataView status check

Also found in 1 other location(s):
- x-pack/solutions/security/plugins/security_solution/public/timelines/components/timeline/tabs/query/events_count.tsx:97 -- The call to `dataView.getRuntimeMappings()` on line 97 does not use optional chaining, unlike other `dataView` accesses in this component (e.g., line 94 uses `dataView?.id`). Before `useDataView` has loaded (when `status` is `'pristine'` or `'loading'`), `dataView` is the initial placeholder value `INITIAL_DV`. If this placeholder doesn't properly implement `getRuntimeMappings()`, the component will crash on initial render. The previous code used `experimentalDataView?.getRuntimeMappings() ?? {}` with both optional chaining and a fallback, which protected against this.

[elasticmachine]

⏳ Build in-progress, with failures

• Buildkite Build
• Commit: 277c76a

Failed CI Steps

• Jest Tests #10
• Jest Tests #10

Test Failures

• [job] [logs] Jest Tests #10 / GraphVisualization entity mode passes a rolling 30-day timeRange
• [job] [logs] Jest Tests #10 / GraphVisualization entity mode passes a rolling 30-day timeRange
• [job] [logs] Jest Tests #10 / GraphVisualization entity mode passes entityIds with isOrigin: true
• [job] [logs] Jest Tests #10 / GraphVisualization entity mode passes entityIds with isOrigin: true
• [job] [logs] Jest Tests #10 / GraphVisualization entity mode renders the wrapper and lazy-loads GraphInvestigation
• [job] [logs] Jest Tests #10 / GraphVisualization entity mode renders the wrapper and lazy-loads GraphInvestigation
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes a timestamp-anchored timeRange
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes a timestamp-anchored timeRange
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes onOpenEventPreview callback
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes onOpenEventPreview callback
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes originEventIds derived from eventIds and isAlert
• [job] [logs] Jest Tests #10 / GraphVisualization event mode passes originEventIds derived from eventIds and isAlert
• [job] [logs] Jest Tests #10 / GraphVisualization event mode renders the wrapper and lazy-loads GraphInvestigation
• [job] [logs] Jest Tests #10 / GraphVisualization event mode renders the wrapper and lazy-loads GraphInvestigation
• [job] [logs] Jest Tests #10 / GraphVisualization onInvestigateInTimeline calls investigateInTimeline with a valid time range
• [job] [logs] Jest Tests #10 / GraphVisualization onInvestigateInTimeline calls investigateInTimeline with a valid time range
• [job] [logs] Jest Tests #10 / GraphVisualization onInvestigateInTimeline shows a danger toast when time range cannot be parsed
• [job] [logs] Jest Tests #10 / GraphVisualization onInvestigateInTimeline shows a danger toast when time range cannot be parsed

History

• 💔 Build #425249 failed 64a3c29
• 💔 Build #425182 failed 5007233
• 💔 Build #424450 failed b36b9ba
• 💔 Build #424443 failed 01444b7
• 💔 Build #410929 failed b98838b