Skip to content

[Security Solution] Enable Value Reports in ESS#243511

Merged
denar50 merged 44 commits intoelastic:mainfrom
denar50:enable-value-reports-in-ech-with-only-ai-generated-data-forwarded
Dec 2, 2025
Merged

[Security Solution] Enable Value Reports in ESS#243511
denar50 merged 44 commits intoelastic:mainfrom
denar50:enable-value-reports-in-ech-with-only-ai-generated-data-forwarded

Conversation

@denar50
Copy link
Contributor

@denar50 denar50 commented Nov 19, 2025

Summary

Issue: https://github.com/elastic/security-team/issues/14504

A previous PR introduced the EASE Value report and the ability to export it in serverless. This PR makes the report available in ESS and adds logic to export it using the share plugin. The ESS export logic is different from that of serverless because schedule reporting is not available in Serverless yet (ResponseOps plans to add support for it in 9.4).

The reporting is initiated in the client's browser when the user clicks on the "Export report" button, which becomes available once the report data and the cost savings trend insight have been fetched and generated respectively.
The export report button makes a call to the server to generate a PDF for the report and passes the insight and a hash of the report data as parameters (aka "forwarded state").

A headless browser is used to navigate to a special route /app/reportingRedirect that looks up a the corresponding locator locator (in this case, AIValueReportLocatorDefinition) which in turns resolves the URL of the value report (/app/security/reports/ai_value) and the forwarded state to be stored in history.location.state.

The value report page reacts to this state being present and renders itself in "export mode". When the components finish loading, the headless browser takes screenshots of everything that is contained within the value report page, which has a data-shared-items-container attribute attached to it.

Notice that we only forward the insight and the hash of the report data in order to avoid calling an LLM again in the headless browser when the data itself hasn't changed.

image

How to test

  1. Run ESS locally (Elasticsearch and Kibana). Then log in as an admin and navigate to the rules management page (app/security/rules/management) to ensure that all indexes are properly initialized.
  2. Use the Security documents generator (fork) to generate 60 days of alert data. By running: yarn start generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date now
  3. Run Attack Discovery over the 60 days of data that you generated (see the video below)
Screen.Recording.2025-11-19.at.15.08.37.mov
  1. Navigate to the "Value report" page. You can use the link on the left side, or you can go to /app/security/reports/ai_value.

  2. Once the report loads, the "Export report" button should be enabled. Click on it and export it to a PDF. You should see a toast indicating that the export in ongoing and when it is done you should get a toast with a "Download report" button. Click on "Download report" and verify that the downloaded PDF matches the data that you are seeing on the screen.
    Play with it by adjusting the time window in the date picker next to the Export report button.

Screen.Recording.2025-11-19.at.15.15.093.mov

Known issues

  1. The icons next to these headings are clearer in the PDF and therefore hard to see
    PDF
image

Website
image

Pending

  1. Copy adjustments to the value report button
  2. Placement of the value report button in the launchpad
  3. Update the documentation on the restricted access view. See Documents new RBAC for value reports docs-content#3817
  4. Follow the instructions after updating the Elastic Assistant prompt.

@denar50 denar50 changed the title Enable value reports in ech with only ai generated data forwarded [Security Solution] Enable value reports in ESS Nov 19, 2025
@denar50 denar50 self-assigned this Nov 19, 2025
@denar50 denar50 force-pushed the enable-value-reports-in-ech-with-only-ai-generated-data-forwarded branch from 0ce5399 to bda9c51 Compare November 19, 2025 15:10
@denar50 denar50 force-pushed the enable-value-reports-in-ech-with-only-ai-generated-data-forwarded branch from 5a798ee to daacc5d Compare November 19, 2025 15:41
@denar50 denar50 changed the title [Security Solution] Enable value reports in ESS [Security Solution] Enable EASE Value reports in ESS Nov 19, 2025
@denar50 denar50 added the backport:skip This PR does not require backporting label Nov 19, 2025
@denar50 denar50 marked this pull request as ready for review November 21, 2025 15:09
@denar50 denar50 requested review from a team as code owners November 21, 2025 15:09
@denar50 denar50 added the release_note:feature Makes this part of the condensed release notes label Nov 21, 2025
@denar50 denar50 added the ci:cloud-deploy Create or update a Cloud deployment label Nov 26, 2025
@denar50 denar50 requested a review from stephmilovic November 26, 2025 14:54
@denar50 denar50 requested a review from vitaliidm November 27, 2025 13:50
Copy link
Contributor

@nreese nreese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes to deeplinks package LGTM
code review only

Copy link
Contributor

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ResponseOps changes (so far) LGTM, but there's a question about getting telemetry from the headless browser, so we may need to re-look at this if more changes are required.

"description": "Non-default value of setting."
}
},
"securitySolution:defaultValueReportMinutes": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to look out for, is if somehow the telemetry index isn't rolled over to take into account this field, and it's passed as a number, the mapping will end up being a numeric type, not keyword. Not sure how the telemetry mapping is handled, but I believe we've had issues like this in the past.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are ways to define the type in the telemetry index either by mapping it in the telemetry index config file (one of these files). Some of these configs also use component templates.

I checked for stack_management in both places and didn't see any place where types have been explicitly defined. In that case it'll depend on the index's dynamic mapping config. I'm not sure in which index these fields will go so I can't say exactly how dynamic is defined, but there are definitely ways to deal with typing the fields and even handling cases where a field's type changes over time. Feel free to reach out in Slack at #platform-analytics if you have questions.

@kibanamachine
Copy link
Contributor

🤖 Prompt Changes Detected

Changes have been detected to one or more prompt files in the Elastic Assistant plugin.

Please remember to update the integrations repository with your prompt changes to ensure consistency across all deployments.

Next Steps:

  1. Follow the documentation in x-pack/solutions/security/packages/security-ai-prompts/README.md to update the corresponding prompt files
  2. Make the changes in the integrations repository
  3. Test your changes in the integrations environment
  4. Ensure prompt consistency across all deployments

This is an automated reminder to help maintain prompt consistency across repositories.

@elasticmachine
Copy link
Contributor

elasticmachine commented Dec 2, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 8436 8443 +7

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/deeplinks-analytics 11 12 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 2.1MB 2.1MB +26.0B
securitySolution 11.1MB 11.1MB -58.6KB
total -58.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
reporting 41.6KB 41.6KB +60.0B
securitySolution 106.9KB 166.5KB +59.6KB
total +59.7KB
Unknown metric groups

API count

id before after diff
@kbn/deeplinks-analytics 11 12 +1

ESLint disabled line counts

id before after diff
securitySolution 699 698 -1

Total ESLint disabled count

id before after diff
securitySolution 807 806 -1

History

cc @denar50

Copy link
Contributor

@vitaliidm vitaliidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detection engine area LGTM

I still think we should investigate how not to set value report urls on visiting random security page, but only when user visits value report page.

@stephmilovic
Copy link
Contributor

I still think we should investigate how not to set value report urls on visiting random security page, but only when user visits value report page.

I understand the concern, but this behavior is consistent with how the app already works. Both global and timeline time ranges are always stored in the URL, even when the user is not on the corresponding pages, because the URL acts as the persistent state container across the entire app. valueReport is being added as a third time range under the same pattern.

If we were to only inject valueReport into the URL when the user specifically visits the value report page, that would break uniformity in how time state is stored and restored. Changing that behavior would require redefining the entire state storage approach for all time ranges, not just this one. That would mean designing a replacement mechanism for global, timeline, and now valueReport.

This URL based state storage approach was chosen originally because it keeps time scoped navigation consistent, predictable, and shareable. If there is an interest in changing the paradigm, that would need to be evaluated holistically rather than selectively for one of the time ranges.

Copy link
Contributor

@stephmilovic stephmilovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work on this @denar50, not an easy challenge and you handled it with grace. Well done! LGTM

Copy link

@leathekd leathekd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

telemetry schema LGTM

@denar50 denar50 merged commit 954102b into elastic:main Dec 2, 2025
12 checks passed
NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request Dec 2, 2025
## Summary
Issue: elastic/security-team#14504

[A previous PR](elastic#228877)
introduced the EASE Value report and the ability to export it in
serverless. This PR makes the report available in ESS and adds logic to
export it using the share plugin. The ESS export logic is different from
that of serverless because schedule reporting is not available in
Serverless yet (ResponseOps
[plans](elastic/kibana-team#1847) to add
support for it in 9.4).

The reporting is initiated in the client's browser when the user clicks
on the "Export report" button, which becomes available once the report
data and the cost savings trend insight have been fetched and generated
respectively.
The export report button makes a call to the server to generate a PDF
for the report and passes the insight and a hash of the report data as
parameters (aka "forwarded state").

A headless browser is used to navigate to a special route
`/app/reportingRedirect` that looks up a the corresponding locator
locator (in this case, `AIValueReportLocatorDefinition`) which in turns
resolves the URL of the value report (`/app/security/reports/ai_value`)
_and_ the forwarded state to be stored in `history.location.state`.

The value report page reacts to this state being present and renders
itself in "export mode". When the components finish loading, the
headless browser takes screenshots of everything that is contained
within the value report page, which has a `data-shared-items-container`
attribute attached to it.

Notice that we only forward the insight and the hash of the report data
in order to avoid calling an LLM again in the headless browser when the
data itself hasn't changed.

<img width="2766" height="948" alt="image"
src="https://github.com/user-attachments/assets/01fbac58-1450-42e0-a16d-c456e9137878"
/>


## How to test
1. Run ESS locally (Elasticsearch and Kibana). Then log in as an admin
and navigate to the rules management page
(`app/security/rules/management`) to ensure that all indexes are
properly initialized.
2. Use the [Security documents
generator](https://github.com/enriquesanchez-elastic/security-documents-generator)
(fork) to generate 60 days of alert data. By running: `yarn start
generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date now`
3. Run Attack Discovery over the 60 days of data that you generated (see
the video below)


https://github.com/user-attachments/assets/85cdefe8-2fc0-4a9c-ab7c-051ca7188b6f

4. Navigate to the "Value report" page. You can use the link on the left
side, or you can go to `/app/security/reports/ai_value`.

5. Once the report loads, the "Export report" button should be enabled.
Click on it and export it to a PDF. You should see a toast indicating
that the export in ongoing and when it is done you should get a toast
with a "Download report" button. Click on "Download report" and verify
that the downloaded PDF matches the data that you are seeing on the
screen.
Play with it by adjusting the time window in the date picker next to the
Export report button.



https://github.com/user-attachments/assets/e1aea0ad-4b74-4ee3-b329-43181d479328


## Known issues
1. The icons next to these headings are clearer in the PDF and therefore
hard to see
PDF
<img width="2856" height="254" alt="image"
src="https://github.com/user-attachments/assets/849b465b-2efd-4f5d-a907-e5570639e2bd"
/>

Website
<img width="1814" height="166" alt="image"
src="https://github.com/user-attachments/assets/9de8da9d-4779-4495-8477-3803d0a3e8e4"
/>

## Pending
1. Copy adjustments to the value report button
2. Placement of the value report button in the launchpad
3. Update the documentation on the restricted access view. See
elastic/docs-content#3817
4. Follow the instructions after updating the Elastic Assistant prompt.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
JordanSh pushed a commit to JordanSh/kibana that referenced this pull request Dec 9, 2025
## Summary
Issue: elastic/security-team#14504

[A previous PR](elastic#228877)
introduced the EASE Value report and the ability to export it in
serverless. This PR makes the report available in ESS and adds logic to
export it using the share plugin. The ESS export logic is different from
that of serverless because schedule reporting is not available in
Serverless yet (ResponseOps
[plans](elastic/kibana-team#1847) to add
support for it in 9.4).

The reporting is initiated in the client's browser when the user clicks
on the "Export report" button, which becomes available once the report
data and the cost savings trend insight have been fetched and generated
respectively.
The export report button makes a call to the server to generate a PDF
for the report and passes the insight and a hash of the report data as
parameters (aka "forwarded state").

A headless browser is used to navigate to a special route
`/app/reportingRedirect` that looks up a the corresponding locator
locator (in this case, `AIValueReportLocatorDefinition`) which in turns
resolves the URL of the value report (`/app/security/reports/ai_value`)
_and_ the forwarded state to be stored in `history.location.state`.

The value report page reacts to this state being present and renders
itself in "export mode". When the components finish loading, the
headless browser takes screenshots of everything that is contained
within the value report page, which has a `data-shared-items-container`
attribute attached to it.

Notice that we only forward the insight and the hash of the report data
in order to avoid calling an LLM again in the headless browser when the
data itself hasn't changed.

<img width="2766" height="948" alt="image"
src="https://github.com/user-attachments/assets/01fbac58-1450-42e0-a16d-c456e9137878"
/>


## How to test
1. Run ESS locally (Elasticsearch and Kibana). Then log in as an admin
and navigate to the rules management page
(`app/security/rules/management`) to ensure that all indexes are
properly initialized.
2. Use the [Security documents
generator](https://github.com/enriquesanchez-elastic/security-documents-generator)
(fork) to generate 60 days of alert data. By running: `yarn start
generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date now`
3. Run Attack Discovery over the 60 days of data that you generated (see
the video below)


https://github.com/user-attachments/assets/85cdefe8-2fc0-4a9c-ab7c-051ca7188b6f

4. Navigate to the "Value report" page. You can use the link on the left
side, or you can go to `/app/security/reports/ai_value`.

5. Once the report loads, the "Export report" button should be enabled.
Click on it and export it to a PDF. You should see a toast indicating
that the export in ongoing and when it is done you should get a toast
with a "Download report" button. Click on "Download report" and verify
that the downloaded PDF matches the data that you are seeing on the
screen.
Play with it by adjusting the time window in the date picker next to the
Export report button.



https://github.com/user-attachments/assets/e1aea0ad-4b74-4ee3-b329-43181d479328


## Known issues
1. The icons next to these headings are clearer in the PDF and therefore
hard to see
PDF
<img width="2856" height="254" alt="image"
src="https://github.com/user-attachments/assets/849b465b-2efd-4f5d-a907-e5570639e2bd"
/>

Website
<img width="1814" height="166" alt="image"
src="https://github.com/user-attachments/assets/9de8da9d-4779-4495-8477-3803d0a3e8e4"
/>

## Pending
1. Copy adjustments to the value report button
2. Placement of the value report button in the launchpad
3. Update the documentation on the restricted access view. See
elastic/docs-content#3817
4. Follow the instructions after updating the Elastic Assistant prompt.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
@florent-leborgne florent-leborgne added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Jan 9, 2026
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:feature Makes this part of the condensed release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.