[8.12] [Security Solution] [Elastic AI Assistant] Include acknowledged alerts in the LangChain AlertCountsTool aggregation (#173701)#173801
Merged
andrew-goldstein merged 1 commit intoelastic:8.12from Dec 21, 2023
Conversation
…rts in the LangChain `AlertCountsTool` aggregation (elastic#173701) This PR updates the LangChain `AlertCountsTool` aggregation, which answers questions like `How many open alerts do I have?`, to include `acknowledged` alerts. The `AlertCountsTool` was introduced as part of [[Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts elastic#172542](elastic#172542) - This PR is similar to <elastic#173121>, where `acknowledged` alerts were added to the `OpenAndAcknowledgedAlertsTool`, which returns the _details_ of alerts - In contrast to [elastic#173121](elastic#173121), this PR is focused on the alert counts _aggregation_ - This PR also updates the `range` of **both** the `AlertCountsTool` and the `OpenAndAcknowledgedAlertsTool` queries to standardize on the following syntax, which aligns with the `Last 24 hours` option in the _Commonly used_ section of the Kibana date picker: ```json "range": { "@timestamp": { "gte": "now-24h", "lte": "now" } } ``` To desk test this change: - The `assistantRagOnAlerts` feature flag described in [elastic#172542](elastic#172542) must be enabled, per the following example: ``` xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts'] ``` - The `Alerts` feature must be enabled in the assistant settings, per the screenshot below:  1) Generate alerts with a variety of severity (e.g. `low`, `medium`, `high`, and `critical`) 2) After the alerts have been generated, disable all detection rules to keep the counts static during testing 3) Navigate to Security > Alerts 4) Select `Last 24 hours` from the _Commonly used_ section of the global date picker 5) Click the `Treemap` button to select the Treemap visualization 6) In the Treemap's `Group by` input, enter `kibana.alert.severity` 7) Next, in the Treemap's `Group by top` input, enter `kibana.alert.workflow_status` 8) Click the `AI Assistant` button to open the assistant 9) Click the `X` button to clear the conversation 10) Close the assistant 11) Add the following two fields as columns to the Alerts page table: ``` kibana.alert.workflow_status _id ``` 12) Sort the Alerts table, first by `kibana.alert.risk_score` from high to low, and then by `@timestamp` from new to old, per the screenshot below:  **Expected results** - The alerts page date range is `Last 24 hours` - The `Treemap` is selected - The treemap is grouped by `kibana.alert.severity` and then `kibana.alert.workflow_status` - The alerts table has custom sorting and columns, per the screenshot below:  13) Click the `AI Assistant` button to open the assistant 14) Ask the assistant: ``` How many open alerts do I have? ``` **Expected results** - The assistant will report on the counts and workflow status of alerts, per the example response and screenshot below: ``` You have a total of 47 open alerts. Here's the breakdown: 24 alerts with low severity, 12 alerts with medium severity, 7 alerts with high severity, and 4 alerts with critical severity. ```  15) Make note of the counts shown in the assistant, then close the assistant Expected result: - The counts from the assistant match the counts in the treemap legend, per the example screenshot below:  16) Change the workflow status of an alert in the Alerts table from `open` to `acknowledged` **Expected result** - The treemap and alerts table and include the updated (`acknowledged`) alert, per the screenshot below:  17) Once again, open the assistant 18) Once again, ask the (same) question: ``` How many open alerts do I have? ``` **Expected result** - The response from the assistant makes reference to the alert who's workflow status was changed from `open` to `acknowledged`, per the example response and screenshot below: ``` Based on the latest data I had received, you have a total of 47 open alerts. Here's the breakdown: 24 alerts are of low severity, 12 alerts are of medium severity, 7 alerts are of high severity, and 4 alerts are of critical severity (Note: One of the critical severity alerts has been acknowledged). ```  (cherry picked from commit 081f52b)
Contributor
Author
|
This backport was desk tested locally in the
|
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]
To update your PR or re-run it, just comment with: |
stephmilovic
approved these changes
Dec 21, 2023
Contributor
stephmilovic
left a comment
stephmilovic
left a comment
There was a problem hiding this comment.
LGTM! Thanks for the screenshots from your desk testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.12:acknowledgedalerts in the LangChainAlertCountsToolaggregation (#173701)Questions ?
Please refer to the Backport tool documentation