[AI4DSOC] Alert Flyout

[stephmilovic]

Summary

Alert flyout for AI for the SOC.

The flyout sections include:

• New header highlighting the integration source
  

• AI generated alert summary generated by button (Generate or Regenerate). Stored in a new data stream (.kibana-elastic-ai-assistant-alert-summary-*)
  

• Anonymization toggle for the alert summary is located in the flyout gear settings menu
  

• Highlighted fields
  

• Attack discovery MiniAttackChain (currently hardcoded to a preconfigured connector, waiting for further work from @andrew-goldstein to hook up to actual alert related AD)
  

• Conversations dropdown that show any conversations this alert is referenced
  

• Suggested prompts that create a new conversation with the alert as context (copy pending)
  

• The connector used in the alert summary generation is selected in Stack Management > Advanced Settings > Security Solution > Default AI Connector (copy pending)
  

New prompts

This PR adds 2 new prompts under a new promptGroupId.aiForSoc:
- promptDictionary.alertSummarySystemPrompt
- promptDictionary.alertSummary
In order to access these prompts in the proper spots, the new find alert summary route returns the "user" prompt (promptDictionary.alertSummary). In order to get the system prompt in place, we pass a promptIds object to the POST_ACTIONS_CONNECTOR_EXECUTE which is appended to the main system prompt

Testing

This needs to be ran in Serverless:

• yarn es serverless --projectType security
• yarn serverless-security --no-base-path

You also need to enable the AI for SOC tier, by adding the following to your serverless.security.dev.yml file:

xpack.securitySolutionServerless.productTypes:
  [
    { product_line: 'ai_soc', product_tier: 'search_ai_lake' },
  ]

Use one of these Serverless users:

• platform_engineer
• endpoint_operations_analyst
• endpoint_policy_manager
• admin
• system_indices_superuser

Then:

• generate data: yarn test:generate:serverless-dev
• create 4 catch all rules, each with a name of a AI for SOC integration (google_secops, microsoft_sentinel,, sentinel_one and crowdstrike) => to do that you'll need to temporary comment the serverless.security.dev.yaml config changes as the rules page is not accessible in AI for SOC.
• change this line to installedPackages: availablePackages to force having some packages installed
• change this line to r.name === p.name to make sure there will be matches between integrations and rules

With this alerts data, you should be able to test each section of the flyout except the attack discovery widget, instructions for that are below.

Attack discovery widget

As I am waiting for updates from Andrew, currently the attack discovery widget looks up attack discoveries from a particular preconfigured connector. In order to test:

1. Add preconfigured connector to your kibana.dev.yml: https://p.elstc.co/paste/J2qmGMeQ#GKSPhlggX4F93aUSKJsKpsqtCcyTepCkfJOEVxlZyfB
2. Generate attack discovery with this connector
3. Open the new flyout, you will see the attack discovery widget

Outstanding TODOs

These are all noted in the code

1. Attack discovery widget is hardcoded to the preconfigured connector id. The widget should instead look up discoveries by alert ID, pending work from @andrew-goldstein
2. Update copy for suggested prompts
3. Update copy for ai connector UI setting
4. Update AI connector UI setting to default to Elastic Managed LLM once it is fully available in serverless

[PhilippeOberti]

I just noticed a wrong usage of DocumentDetailsContextProvider, I'm blocking the PR just to be sure and I'm working on replacing it with the AIForSocContextProvider. This requires making some changes to the HighlightedFields component and will require testing in the alerts page flyout

[PhilippeOberti]

LGTM for the Threat Hunting Investigations team!

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: aba0c21

Failed CI Steps

• Defend Workflows Cypress Tests on Serverless #14

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
automaticImport	731	752	+21
securitySolution	7234	7259	+25
total			+46

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/elastic-assistant	141	145	+4
@kbn/elastic-assistant-common	517	526	+9
total			+13

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
advancedSettings	36.8KB	37.1KB	+331.0B
apm	2.5MB	2.5MB	+331.0B
infra	1.2MB	1.2MB	+331.0B
observabilityAiAssistantManagement	81.0KB	81.3KB	+331.0B
profiling	372.8KB	373.1KB	+331.0B
securitySolution	9.0MB	9.0MB	+26.9KB
telemetryManagementSection	31.4KB	31.8KB	+329.0B
total			+28.8KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
@kbn/elastic-assistant-common	0	1	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	89.4KB	89.5KB	+46.0B

Unknown metric groups

API count

id	before	after	diff
@kbn/elastic-assistant	170	178	+8
@kbn/elastic-assistant-common	606	617	+11
total			+19

ESLint disabled line counts

id	before	after	diff
@kbn/elastic-assistant	13	14	+1
elasticAssistant	34	37	+3
total			+4

Total ESLint disabled count

id	before	after	diff
@kbn/elastic-assistant	14	15	+1
elasticAssistant	39	42	+3
total			+4

History

• 💚 Build #294079 succeeded f6a9f67
• 💔 Build #293970 failed 52a78c9
• 💔 Build #293965 failed ec1202d
• 💔 Build #293907 failed d2ab93a
• 💔 Build #293878 failed c67ed18

[PhilippeOberti]

run docs-build

The code that was modifying the alerts_context.tsx file has been reverted as it is not needed anymore.

[PhilippeOberti]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation