[Security Solution] Enable Value Reports in ESS#243511
[Security Solution] Enable Value Reports in ESS#243511denar50 merged 44 commits intoelastic:mainfrom
Conversation
0ce5399 to
bda9c51
Compare
5a798ee to
daacc5d
Compare
…erated-data-forwarded
…erated-data-forwarded
…erated-data-forwarded
…s-in-ech-with-only-ai-generated-data-forwarded
…erated-data-forwarded
…erated-data-forwarded
nreese
left a comment
There was a problem hiding this comment.
changes to deeplinks package LGTM
code review only
pmuellr
left a comment
There was a problem hiding this comment.
ResponseOps changes (so far) LGTM, but there's a question about getting telemetry from the headless browser, so we may need to re-look at this if more changes are required.
| "description": "Non-default value of setting." | ||
| } | ||
| }, | ||
| "securitySolution:defaultValueReportMinutes": { |
There was a problem hiding this comment.
One thing to look out for, is if somehow the telemetry index isn't rolled over to take into account this field, and it's passed as a number, the mapping will end up being a numeric type, not keyword. Not sure how the telemetry mapping is handled, but I believe we've had issues like this in the past.
There was a problem hiding this comment.
There are ways to define the type in the telemetry index either by mapping it in the telemetry index config file (one of these files). Some of these configs also use component templates.
I checked for stack_management in both places and didn't see any place where types have been explicitly defined. In that case it'll depend on the index's dynamic mapping config. I'm not sure in which index these fields will go so I can't say exactly how dynamic is defined, but there are definitely ways to deal with typing the fields and even handling cases where a field's type changes over time. Feel free to reach out in Slack at #platform-analytics if you have questions.
…erated-data-forwarded
🤖 Prompt Changes DetectedChanges have been detected to one or more prompt files in the Elastic Assistant plugin. Please remember to update the integrations repository with your prompt changes to ensure consistency across all deployments. Next Steps:
This is an automated reminder to help maintain prompt consistency across repositories. |
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
Unknown metric groupsAPI count
ESLint disabled line counts
Total ESLint disabled count
History
cc @denar50 |
vitaliidm
left a comment
There was a problem hiding this comment.
Detection engine area LGTM
I still think we should investigate how not to set value report urls on visiting random security page, but only when user visits value report page.
I understand the concern, but this behavior is consistent with how the app already works. Both If we were to only inject This URL based state storage approach was chosen originally because it keeps time scoped navigation consistent, predictable, and shareable. If there is an interest in changing the paradigm, that would need to be evaluated holistically rather than selectively for one of the time ranges. |
stephmilovic
left a comment
There was a problem hiding this comment.
Amazing work on this @denar50, not an easy challenge and you handled it with grace. Well done! LGTM
## Summary Issue: elastic/security-team#14504 [A previous PR](elastic#228877) introduced the EASE Value report and the ability to export it in serverless. This PR makes the report available in ESS and adds logic to export it using the share plugin. The ESS export logic is different from that of serverless because schedule reporting is not available in Serverless yet (ResponseOps [plans](elastic/kibana-team#1847) to add support for it in 9.4). The reporting is initiated in the client's browser when the user clicks on the "Export report" button, which becomes available once the report data and the cost savings trend insight have been fetched and generated respectively. The export report button makes a call to the server to generate a PDF for the report and passes the insight and a hash of the report data as parameters (aka "forwarded state"). A headless browser is used to navigate to a special route `/app/reportingRedirect` that looks up a the corresponding locator locator (in this case, `AIValueReportLocatorDefinition`) which in turns resolves the URL of the value report (`/app/security/reports/ai_value`) _and_ the forwarded state to be stored in `history.location.state`. The value report page reacts to this state being present and renders itself in "export mode". When the components finish loading, the headless browser takes screenshots of everything that is contained within the value report page, which has a `data-shared-items-container` attribute attached to it. Notice that we only forward the insight and the hash of the report data in order to avoid calling an LLM again in the headless browser when the data itself hasn't changed. <img width="2766" height="948" alt="image" src="https://github.com/user-attachments/assets/01fbac58-1450-42e0-a16d-c456e9137878" /> ## How to test 1. Run ESS locally (Elasticsearch and Kibana). Then log in as an admin and navigate to the rules management page (`app/security/rules/management`) to ensure that all indexes are properly initialized. 2. Use the [Security documents generator](https://github.com/enriquesanchez-elastic/security-documents-generator) (fork) to generate 60 days of alert data. By running: `yarn start generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date now` 3. Run Attack Discovery over the 60 days of data that you generated (see the video below) https://github.com/user-attachments/assets/85cdefe8-2fc0-4a9c-ab7c-051ca7188b6f 4. Navigate to the "Value report" page. You can use the link on the left side, or you can go to `/app/security/reports/ai_value`. 5. Once the report loads, the "Export report" button should be enabled. Click on it and export it to a PDF. You should see a toast indicating that the export in ongoing and when it is done you should get a toast with a "Download report" button. Click on "Download report" and verify that the downloaded PDF matches the data that you are seeing on the screen. Play with it by adjusting the time window in the date picker next to the Export report button. https://github.com/user-attachments/assets/e1aea0ad-4b74-4ee3-b329-43181d479328 ## Known issues 1. The icons next to these headings are clearer in the PDF and therefore hard to see PDF <img width="2856" height="254" alt="image" src="https://github.com/user-attachments/assets/849b465b-2efd-4f5d-a907-e5570639e2bd" /> Website <img width="1814" height="166" alt="image" src="https://github.com/user-attachments/assets/9de8da9d-4779-4495-8477-3803d0a3e8e4" /> ## Pending 1. Copy adjustments to the value report button 2. Placement of the value report button in the launchpad 3. Update the documentation on the restricted access view. See elastic/docs-content#3817 4. Follow the instructions after updating the Elastic Assistant prompt. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
## Summary Issue: elastic/security-team#14504 [A previous PR](elastic#228877) introduced the EASE Value report and the ability to export it in serverless. This PR makes the report available in ESS and adds logic to export it using the share plugin. The ESS export logic is different from that of serverless because schedule reporting is not available in Serverless yet (ResponseOps [plans](elastic/kibana-team#1847) to add support for it in 9.4). The reporting is initiated in the client's browser when the user clicks on the "Export report" button, which becomes available once the report data and the cost savings trend insight have been fetched and generated respectively. The export report button makes a call to the server to generate a PDF for the report and passes the insight and a hash of the report data as parameters (aka "forwarded state"). A headless browser is used to navigate to a special route `/app/reportingRedirect` that looks up a the corresponding locator locator (in this case, `AIValueReportLocatorDefinition`) which in turns resolves the URL of the value report (`/app/security/reports/ai_value`) _and_ the forwarded state to be stored in `history.location.state`. The value report page reacts to this state being present and renders itself in "export mode". When the components finish loading, the headless browser takes screenshots of everything that is contained within the value report page, which has a `data-shared-items-container` attribute attached to it. Notice that we only forward the insight and the hash of the report data in order to avoid calling an LLM again in the headless browser when the data itself hasn't changed. <img width="2766" height="948" alt="image" src="https://github.com/user-attachments/assets/01fbac58-1450-42e0-a16d-c456e9137878" /> ## How to test 1. Run ESS locally (Elasticsearch and Kibana). Then log in as an admin and navigate to the rules management page (`app/security/rules/management`) to ensure that all indexes are properly initialized. 2. Use the [Security documents generator](https://github.com/enriquesanchez-elastic/security-documents-generator) (fork) to generate 60 days of alert data. By running: `yarn start generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date now` 3. Run Attack Discovery over the 60 days of data that you generated (see the video below) https://github.com/user-attachments/assets/85cdefe8-2fc0-4a9c-ab7c-051ca7188b6f 4. Navigate to the "Value report" page. You can use the link on the left side, or you can go to `/app/security/reports/ai_value`. 5. Once the report loads, the "Export report" button should be enabled. Click on it and export it to a PDF. You should see a toast indicating that the export in ongoing and when it is done you should get a toast with a "Download report" button. Click on "Download report" and verify that the downloaded PDF matches the data that you are seeing on the screen. Play with it by adjusting the time window in the date picker next to the Export report button. https://github.com/user-attachments/assets/e1aea0ad-4b74-4ee3-b329-43181d479328 ## Known issues 1. The icons next to these headings are clearer in the PDF and therefore hard to see PDF <img width="2856" height="254" alt="image" src="https://github.com/user-attachments/assets/849b465b-2efd-4f5d-a907-e5570639e2bd" /> Website <img width="1814" height="166" alt="image" src="https://github.com/user-attachments/assets/9de8da9d-4779-4495-8477-3803d0a3e8e4" /> ## Pending 1. Copy adjustments to the value report button 2. Placement of the value report button in the launchpad 3. Update the documentation on the restricted access view. See elastic/docs-content#3817 4. Follow the instructions after updating the Elastic Assistant prompt. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
Summary
Issue: https://github.com/elastic/security-team/issues/14504
A previous PR introduced the EASE Value report and the ability to export it in serverless. This PR makes the report available in ESS and adds logic to export it using the share plugin. The ESS export logic is different from that of serverless because schedule reporting is not available in Serverless yet (ResponseOps plans to add support for it in 9.4).
The reporting is initiated in the client's browser when the user clicks on the "Export report" button, which becomes available once the report data and the cost savings trend insight have been fetched and generated respectively.
The export report button makes a call to the server to generate a PDF for the report and passes the insight and a hash of the report data as parameters (aka "forwarded state").
A headless browser is used to navigate to a special route
/app/reportingRedirectthat looks up a the corresponding locator locator (in this case,AIValueReportLocatorDefinition) which in turns resolves the URL of the value report (/app/security/reports/ai_value) and the forwarded state to be stored inhistory.location.state.The value report page reacts to this state being present and renders itself in "export mode". When the components finish loading, the headless browser takes screenshots of everything that is contained within the value report page, which has a
data-shared-items-containerattribute attached to it.Notice that we only forward the insight and the hash of the report data in order to avoid calling an LLM again in the headless browser when the data itself hasn't changed.
How to test
app/security/rules/management) to ensure that all indexes are properly initialized.yarn start generate-alerts -n 10000 -h 100 -u 100 --start-date 60d --end-date nowScreen.Recording.2025-11-19.at.15.08.37.mov
Navigate to the "Value report" page. You can use the link on the left side, or you can go to
/app/security/reports/ai_value.Once the report loads, the "Export report" button should be enabled. Click on it and export it to a PDF. You should see a toast indicating that the export in ongoing and when it is done you should get a toast with a "Download report" button. Click on "Download report" and verify that the downloaded PDF matches the data that you are seeing on the screen.
Play with it by adjusting the time window in the date picker next to the Export report button.
Screen.Recording.2025-11-19.at.15.15.093.mov
Known issues
PDF
Website

Pending