[Security Assistant] Context prompts by stephmilovic · Pull Request #224956 · elastic/kibana

stephmilovic · 2025-06-23T20:06:38Z

Summary

Implements the context prompts within the Security AI Prompts package. These prompts were client side only, so in order to fetch the prompts from the integration, a new find security AI prompts API was introduced. If the API fails, the prompts fallback to a client side i18n'd version. The context prompts include:

Alert/event analysis

Data Quality Dashboard

Rules

Imperfections

Given that this user prompt is executed in a conversation with a connector id, it would be nice to include the connector id in the request in case provider specific prompts are introduced. However, these prompts are currently stored in the assistant provider context. Because of the short turnaround required for this feature (I was asked to implement it shortly before feature freeze), I left the prompts in the provider context and did not implement a connector id as a requirement for the request. It would be nice to refactor this in the future, perhaps as a part of this issue: #219142

Cypress changes

We were checking if the correct user prompt was sent when these prompts are used. However, since it is now part of the Security AI Prompt package and the prompt can be changed out of band, this check could break if the integration is updated with a new prompt. Therefore, I removed this check.

Integrations PR

As this adds new prompts, the @elastic/security-generative-ai reviewer should also review this integrations PR: elastic/integrations#14296
The integrations PR includes new prompts from #224981

elasticmachine · 2025-06-23T20:06:41Z

Pinging @elastic/security-solution (Team: SecuritySolution)

stephmilovic · 2025-06-23T20:08:00Z

.../security/packages/ecs-data-quality-dashboard/impl/data_quality_panel/actions/chat/index.tsx

  const styles = useStyles();
-  const { isAssistantEnabled } = useDataQualityContext();
+  const { isAssistantEnabled, httpFetch, toasts } = useDataQualityContext();
+  const {


I tried for about 45 minutes to add tests for this, but the mocking logic in this package is extremely difficult to follow. Given the approaching feature freeze and lack of time, I ask that @elastic/security-threat-hunting-investigations add tests to this PR or forgive the missing tests.

Sorry to hear that. Seems like your AssistantProvider file causes a circular imports issue that makes it impossible to jest mock components inside of it from outside. Will have a look at it some time later when I have time to pinpoint specifically where it happens.

Nevertheless I added tests and pushed to your branch.

To make this work, unfortunately had to completely mock out NewChat and remove assistant provider which makes it a blackbox from now on in our tests to test anything besides the immediate contract of your exposed components.

Also I spotted a slight uncovered edge case in title, fixed and tested it.

Let me know if it works for you

…-fix'

… context_prompts

christineweng

Investigations changes LGTM. I pinged another team member on data quality tests

dplumlee

Rule management changes LGTM

stephmilovic · 2025-06-23T23:24:22Z

@elasticmachine merge upstream

KDKHD

Code changes look good! Ran locally and saw the API request being made and the prompts from the response being added into the Assistant.

… context_prompts

stephmilovic · 2025-06-24T15:00:27Z

@elasticmachine merge upstream

stephmilovic · 2025-06-24T16:12:57Z

@elasticmachine merge upstream

…-fix'

stephmilovic · 2025-06-24T22:41:02Z

@elasticmachine merge upstream

kibanamachine · 2025-06-25T02:33:08Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15865936770

elasticmachine · 2025-06-25T02:34:02Z

💚 Build Succeeded

Buildkite Build
Commit: 0fa979a

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`automaticImport`	755	758	+3
`cases`	1133	1135	+2
`elasticAssistant`	407	410	+3
`securitySolution`	7764	7768	+4
total			+12

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/elastic-assistant`	174	175	+1
`@kbn/elastic-assistant-common`	660	667	+7
total			+8

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	9.7MB	9.8MB	+57.5KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/elastic-assistant`	12	13	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	147.7KB	94.2KB	-53.5KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/elastic-assistant`	206	208	+2
`@kbn/elastic-assistant-common`	772	779	+7
total			+9

History

kibanamachine · 2025-06-25T02:39:40Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts You might need to backport the following PRs to 8.19: - [ReponseOps][Reporting] Allow users to schedule reports and view schedules list (#224849) - [ObsUX] Remove entities v2 code and Inventory (#224696) - [Observability] [Serverless] Introduce custom roles (#219861) - [AI4DSOC] Fix cases capability used in Security navbar (#222504) - [Cloud Secutity Posture] Add cloud connector support for asset discovery (#221689) - [Cloud Security] Add Fleet Extensions for Cloud Asset Discovery to Enable Custom Onboarding UI (#219001)

Manual backport

To create the backport manually run:

node scripts/backport --pr 224956

Questions ?

Please refer to the Backport tool documentation

(cherry picked from commit 2e384fb) # Conflicts: # x-pack/platform/packages/shared/kbn-elastic-assistant-common/impl/schemas/index.ts # x-pack/platform/plugins/private/translations/translations/fr-FR.json # x-pack/platform/plugins/private/translations/translations/ja-JP.json # x-pack/platform/plugins/private/translations/translations/zh-CN.json # x-pack/solutions/security/plugins/security_solution/public/assistant/provider.tsx # x-pack/solutions/security/plugins/security_solution/public/plugin.tsx

stephmilovic · 2025-06-25T03:18:39Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.19`: - [[Security Assistant] Context prompts (#224956)](#224956)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

KDKHD · 2025-06-25T12:46:46Z

x-pack/solutions/security/plugins/security_solution/public/assistant/provider.tsx

+  const promptContext = useObservable(
+    elasticAssistantSharedState.promptContexts.getPromptContext$(),
+    {}
+  );


How come you needed to add this useObservable hook? Was it not enough to call elasticAssistantSharedState.promptContexts.setPromptContext(PROMPT_CONTEXTS) wheneverPROMPT_CONTEXTS changes?

The useEffect was stack overflowing, crashed the app

stephmilovic added 9 commits June 23, 2025 09:15

wip

ba7767e

Merge branch 'main' into context_prompts

85009d3

why

c80fb1e

undo rename

22886e6

public api

4c64b5f

finished code, needs tests

439f265

rm unused schema

802f51d

use assistant update

7a29142

tests

6386908

stephmilovic added release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Jun 23, 2025

stephmilovic requested review from a team as code owners June 23, 2025 20:06

stephmilovic added the Team:Security Generative AI Security Generative AI label Jun 23, 2025

stephmilovic requested a review from a team as a code owner June 23, 2025 20:06

stephmilovic added backport:version Backport to applied version labels v9.1.0 labels Jun 23, 2025

stephmilovic requested a review from xcrzx June 23, 2025 20:06

stephmilovic added the v8.19.0 label Jun 23, 2025

stephmilovic commented Jun 23, 2025

View reviewed changes

stephmilovic mentioned this pull request Jun 23, 2025

[Security AI Prompts] Add new prompts elastic/integrations#14296

Merged

kibanamachine and others added 4 commits June 23, 2025 20:30

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

542a0d1

…-fix'

fix i18n.

1aad4de

Merge branch 'context_prompts' of github.com:stephmilovic/kibana into…

756423d

… context_prompts

fix import

aec1a6a

christineweng approved these changes Jun 23, 2025

View reviewed changes

fix type

ac7da86

dplumlee approved these changes Jun 23, 2025

View reviewed changes

KDKHD approved these changes Jun 24, 2025

View reviewed changes

stephmilovic and others added 3 commits June 24, 2025 07:20

Merge branch 'main' into context_prompts

01a53fe

Merge branch 'context_prompts' of github.com:stephmilovic/kibana into…

a2facb3

… context_prompts

chore: add tests and fix edge case with chat title

4388476

Merge branch 'main' into context_prompts

54359dd

stephmilovic enabled auto-merge (squash) June 24, 2025 15:38

elasticmachine and others added 2 commits June 24, 2025 17:13

Merge branch 'main' into context_prompts

0497721

Merge branch 'main' of github.com:elastic/kibana into HEAD

e2a7463

KDKHD requested a review from a team as a code owner June 24, 2025 18:38

kibanamachine and others added 3 commits June 24, 2025 19:02

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

de70098

…-fix'

fix linter

5f6c67a

hopefully fix failing API call

dfac35b

elasticmachine and others added 5 commits June 24, 2025 23:41

Merge branch 'main' into context_prompts

9871dd1

Merge branch 'main' into context_prompts

48ed561

fix?!

b3cc308

refetchOnWindowFocus: false

e21d2a0

Merge branch 'main' into context_prompts

0fa979a

stephmilovic merged commit 2e384fb into elastic:main Jun 25, 2025
10 checks passed

stephmilovic mentioned this pull request Jun 25, 2025

[8.19] [Security Assistant] Context prompts (#224956) #225195

Merged

akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Jun 25, 2025

[Security Assistant] Context prompts (elastic#224956)

46e00f3

KDKHD reviewed Jun 25, 2025

View reviewed changes

Conversation

stephmilovic commented Jun 23, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Alert/event analysis

Data Quality Dashboard

Rules

Imperfections

Cypress changes

Integrations PR

Uh oh!

elasticmachine commented Jun 23, 2025

Uh oh!

stephmilovic Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kapral18 Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

christineweng left a comment

Choose a reason for hiding this comment

Uh oh!

dplumlee left a comment

Choose a reason for hiding this comment

Uh oh!

stephmilovic commented Jun 23, 2025

Uh oh!

KDKHD left a comment

Choose a reason for hiding this comment

Uh oh!

stephmilovic commented Jun 24, 2025

Uh oh!

stephmilovic commented Jun 24, 2025

Uh oh!

stephmilovic commented Jun 24, 2025

Uh oh!

Uh oh!

kibanamachine commented Jun 25, 2025

Uh oh!

elasticmachine commented Jun 25, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

Uh oh!

kibanamachine commented Jun 25, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

stephmilovic commented Jun 25, 2025

💚 All backports created successfully

Questions ?

Uh oh!

KDKHD Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stephmilovic Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

stephmilovic commented Jun 23, 2025 •

edited by kibanamachine

Loading

stephmilovic Jun 23, 2025 •

edited

Loading

kapral18 Jun 24, 2025 •

edited

Loading

KDKHD Jun 25, 2025 •

edited

Loading