chore(streams): generate significant event definitions #227126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

kdelemme merged 70 commits into elastic:main from kdelemme:streams/sig-events-suggestion

Aug 4, 2025

Contributor

kdelemme commented Jul 8, 2025 •

edited

Loading

Summary

Resolves #214436

Mostly copied over from https://github.com/elastic/kibana/pull/216150/files#diff-51505aa38745af6aa946a2f13b7319659ead2bda2c9d2effc5e061bc88f15b69

This PR adds a /significant_events/_generate API that returns generated significant event definitions based on the current streams data.
The current algorithm works as follow:

Fetch 24 hours of sampled data from the stream name index
Select either "message" or "body.text" as field to run log patterns analysis upon
Run log patterns analysis for the past 7 days on the selected field to find recurring patterns
Ask the LLM to generate significant events queries based on this recurrent patterns
Validate the generated queries by looking for document count in the past 7 days
Ask the LLM to select the queries it thinks are the most important

In this PR we also integrate with the LLM eval runner framework: example

Prompts

initial
followup

Manual testing

Enable streams: POST kbn:/api/streams/_enable
Enable Significant Events advanced settings UI
Run logs generator: node scripts/synthtrace.js sample_logs --live --kibana=http://elastic:changeme@localhost:5601 --target=http://elastic:changeme@localhost:9200 --liveBucketSize=1000
Configure a llm connector: https://ela.st/o11y-ai-openai-credentials

Call the generate significant events API:

curl --request GET \
  --url 'http://localhost:5601/api/streams/logs/significant_events/_generate?connectorId=azure-gpt4o' \
  --header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
  --header 'Content-Type: multipart/form-data' \
  --header 'kbn-xsrf: oui'

(optional) Configure phoenix so you can see the llm prompts

LLM-based testing

Start scout:

node scripts/scout.js start-server --stateful

Run the tests:

node scripts/playwright test --config x-pack/platform/packages/private/kbn-evals-suite-streams/playwright.config.ts

kdelemme added 10 commits

July 7, 2025 18:27


          Dummy generate route

8a44044


          Copy from poc

237bcfb


          Create new genai util package

d5ab990


          Move common files into new shared package

949101f


          Add es query helpers

5d1184f


          Move server files into new shared package

7b4a68e


          Fix connectorId

e840a3a


          Fix refactor

c924393


          Merge branch 'main' into streams/sig-events-suggestion

eaec4ed


          Tweaks

5fb0f22

github-actions bot added the author:obs-ux-management label

kibanamachine added 5 commits

July 8, 2025 19:06


          [CI] Auto-commit changed files from 'node scripts/generate codeowners'

d2939e5


          [CI] Auto-commit changed files from 'node scripts/capture_oas_snapsho…

1f1a03c

…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/streams --include-path /api/fleet --include-path /api/dashboards --include-path /api/saved_objects/_import --include-path /api/saved_objects/_export --include-path /api/maintenance_window --update'


          [CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

3aec274

…no-cache --fix'


          [CI] Auto-commit changed files from 'node scripts/notice'

8cdd845


          [CI] Auto-commit changed files from 'ts-node .buildkite/pipeline-reso…

748b481

…urce-definitions/scripts/fix-location-collection.ts'

kdelemme added release_note:skip Team:actionable-obs backport:version v9.2.0 labels

kdelemme self-assigned this

kibanamachine and others added 8 commits

July 8, 2025 20:31


          [CI] Auto-commit changed files from 'make api-docs'

e4caa53


          Fix linter

902a8f9


          [CI] Auto-commit changed files from 'node scripts/styled_components_m…

e55446b

…apping'


          Fix linter;

16fcdf7


          Merge branch 'main' into streams/sig-events-suggestion

8b3e78c


          fix server package

ba44701


          remove dedent

5c4bfb7


          Update response

34dbfbb

kdelemme marked this pull request as ready for review

July 9, 2025 14:58

kdelemme removed the request for review from lukasolson

August 4, 2025 18:00

Contributor

elasticmachine commented Aug 4, 2025

💔 Build Failed

Buildkite Build
Commit: c12217d

Failed CI Steps

Check Types

History

💛 Build #326057 was flaky 3156242
💚 Build #325745 succeeded b5ca970
💚 Build #325599 succeeded 259036d
💔 Build #325563 failed acc46a5
💔 Build #325314 failed 8ba73ca

kdelemme force-pushed the streams/sig-events-suggestion branch from c12217d to 3156242 Compare

August 4, 2025 18:45

lukasolson approved these changes

View reviewed changes

kdelemme merged commit 0c13652 into elastic:main

18 checks passed

kdelemme deleted the streams/sig-events-suggestion branch

August 4, 2025 19:36

szaffarano pushed a commit to szaffarano/kibana that referenced this pull request


          chore(streams): generate significant event definitions (elastic#227126)

d14ec09

delanni pushed a commit to delanni/kibana that referenced this pull request


          chore(streams): generate significant event definitions (elastic#227126)

030ab14

kibanamachine added the backport missing label

Contributor

kibanamachine commented Aug 6, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

wildemat mentioned this pull request

pr 230826 #231022

Closed

10 tasks

Contributor

kibanamachine commented Aug 7, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

7 similar comments

Contributor

kibanamachine commented Aug 8, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 8, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 11, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 12, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 13, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 14, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 15, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request


          chore(streams): generate significant event definitions (elastic#227126)

e759a6f

Contributor

kibanamachine commented Aug 18, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

6 similar comments

Contributor

kibanamachine commented Aug 20, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 21, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 22, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 25, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 26, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

Contributor

kibanamachine commented Aug 27, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 227126 locally
cc: @kdelemme

maryam-saeidi added backport:skip and removed backport:version labels

kibanamachine removed the backport missing label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

lukasolson lukasolson approved these changes

jbudz jbudz approved these changes

florent-leborgne florent-leborgne approved these changes

csr csr approved these changes

+1 more reviewer

dgieselaar dgieselaar approved these changes

Labels

author:obs-ux-management backport:skip ci:project-deploy-observability release_note:skip Team:actionable-obs v9.2.0