[SigEvents] Seed sigevents env script by crespocarlos · Pull Request #261172 · elastic/kibana

crespocarlos · 2026-04-03T17:07:36Z

Summary

What it seeds:

Logs via SynthtraceClientsManager + @kbn/synthtrace-client timerange API into the synthtrace default stream (logs-synth-default), with a 30-minute baseline window followed by a failure injection phase
Features (entity/service, entity/infra, dependency edges) into .kibana_streams_features-*
Queries (ES|QL rules) via the Streams queries API, with post-promotion rule_id resolution via a bounded poll against .kibana_streams_assets
Alerts (one per matching log row per query) into .alerts-streams.alerts-default, with deterministic IDs derived via UUIDv5
Insights via /internal/streams/_insights/_bulk
Task docs (features, queries, onboarding, insights) into .kibana_streams_tasks via a temporary system_indices_superuser account
Key design points:
All seeded IDs are deterministic (UUIDv5) — re-running produces the same IDs, making the seed idempotent
--clean wipes all seeded data before re-seeding
Shared payload builders (buildFeaturePayloads, buildInsightPayloads) live in lib/builders.ts and are consumed by both the seed steps and the task doc assembler, keeping payloads consistent across storage paths
Stream name is always derived from synthtrace's own routing (getSynthtraceDefaultStream()) — no hardcoding

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

🤖 Co-authored with AI assistance.

…gevents-env

…trace usage

macroscopeapp · 2026-04-06T21:07:54Z

Approvability

Verdict: Needs human review

This PR introduces a substantial new development seeding tool (~1400 lines) for the sigevents environment. While it's dev-only tooling that doesn't affect production runtime, the author does not own any of the changed files - all are owned by @elastic/obs-onboarding-team and @elastic/obs-sig-events-team. The designated code owners should review this new capability.

^{You can customize Macroscope's approvability policy. Learn more.}

ruflin · 2026-04-08T11:04:05Z

Did a quick test run and this works as expected for KI and queries. It did not generated any insights / Sig events for me.

We had a quick discussion around that it would be nice, if we could also specify some specific cases we want to see / test in the UI here which is not directly the case. But my understanding is we can modify / enhance the generation at any time to also new data structures inside, all the tooling is in place.

crespocarlos · 2026-04-08T11:53:37Z

Did a quick test run and this works as expected for KI and queries. It did not generated any insights / Sig events for me.

It should have generated insights. let me test it again

We had a quick discussion around that it would be nice, if we could also specify some specific cases we want to see / test in the UI here which is not directly the case. But my understanding is we can modify / enhance the generation at any time to also new data structures inside, all the tooling is in place.

what specific cases would that be?

taskDocIds included INSIGHTS_TASK_TYPE (global singleton) alongside per-stream IDs, meaning cleanTasks would delete the global insights task whenever any single stream was cleaned — breaking other streams if multiple were seeded simultaneously. Rename taskDocIds → streamTaskDocIds (per-stream only), resolve insightsId directly from INSIGHTS_TASK_TYPE in buildTaskDocs, and assemble the full ID list explicitly in cleanTasks with a comment documenting the multi-stream trade-off. Made-with: Cursor

ruflin · 2026-04-08T14:16:17Z

Tested again and insights are generated now.

what specific cases would that be?

I'm thinking of cases like having on Significant Events with a too long title, missing fields etc. All the fun that LLMs generate from time to time should not break the UI.

crespocarlos · 2026-04-08T16:09:30Z

@elasticmachine merge upstream

crespocarlos · 2026-04-09T15:04:03Z

@elasticmachine merge upstream

ruflin · 2026-04-10T11:06:03Z

Tested again and works as expected. We can add more and other data entries over time.

crespocarlos · 2026-04-14T11:59:53Z

@copilot resolve the merge conflicts in this pull request

crespocarlos · 2026-04-14T12:33:11Z

@elasticmachine merge upstream

elasticmachine · 2026-04-14T12:33:14Z

merge conflict between base and head

crespocarlos · 2026-04-14T12:36:16Z

@elasticmachine merge upstream

elasticmachine · 2026-04-14T12:36:20Z

There are no new commits on the base branch.

elasticmachine · 2026-04-14T13:56:43Z

💚 Build Succeeded

Buildkite Build
Commit: 507bce5

Metrics [docs]

✅ unchanged

History

💛 Build #424920 was flaky 27c6f48
💔 Build #424154 failed d8aa8b6
💔 Build #424021 failed 8a19c29
💛 Build #422106 was flaky 250e38c
💔 Build #422081 failed 0a389b2

* commit '11ed3645c5ededae2a6e29f2a79b31f52208b441': (157 commits) remove sync register uiAction methods (elastic#254590) [performance] Apply minimal auth to the search route (elastic#257497) [ES|QL] Reports correctly the controls server side errors (elastic#263020) [SecuritySolution][Navigation] Enable classic nav updates (elastic#262358) [Inference] Use pretty name and logo on feature settings page (elastic#262531) [Security Solution] fix AT-AB cypress test (elastic#262991) [SigEvents] Seed sigevents env script (elastic#261172) Adjust conditions for validating no refetch for expanded row (elastic#262978) [Agent Builder] update copy for the announcement modal (elastic#263034) [Search] Hide index management links for users without privileges (elastic#262627) Simplify OAS schema for GET `/api/spaces/space` query params (elastic#260831) Fix fleet output OAS regressions: SSL type explosion and Kafka union wrappers (elastic#260842) [Dashboards in chat] fix agent confusing the axes in a horizontal chat (elastic#263064) [One Workflow] Add alert state checkbox UI for workflow connector (elastic#259770) [One Workflow] Deprecate legacy Cases step types in workflow authoring (elastic#262070) skip failing test suite (elastic#248090) fix flaky test: MonitorDetails filter apply button not enabled (elastic#260788) fix: propagate AbortSignal to executeAsReasoningAgent for task cancellation (elastic#262811) [Security Solution][Alert KPI] Fix white space bug in alert KPIs (elastic#260803) [Streams] Move helpers and format_size_unit to utils folder (elastic#262550) ... # Conflicts: # x-pack/platform/plugins/shared/dashboard_agent/public/attachment_types/canvas_integration/dashboard_canvas_content.test.tsx # x-pack/platform/plugins/shared/dashboard_agent/public/attachment_types/canvas_integration/dashboard_canvas_content.tsx # x-pack/platform/plugins/shared/dashboard_agent/public/attachment_types/canvas_integration/use_register_canvas_action_buttons.ts # x-pack/platform/plugins/shared/dashboard_agent/public/attachment_types/index.test.tsx # x-pack/platform/plugins/shared/dashboard_agent/public/attachment_types/index.tsx

feat(streams): add seed_sigevents_env local dev seeder script

0a389b2

crespocarlos force-pushed the feature/seed-sigevents-env branch from 61431b3 to 0a389b2 Compare April 3, 2026 17:18

kibanamachine and others added 3 commits April 3, 2026 17:43

Changes from node scripts/telemetry_check

ebe4ebb

Changes from node scripts/regenerate_moon_projects.js --update

250e38c

refactor(streams): migrate seeder to dev-cli-runner, simplify cleanup

34706e2

crespocarlos changed the title ~~Feature/seed sigevents env~~ [SigEvents] Seed sigevents env script Apr 6, 2026

crespocarlos added 3 commits April 6, 2026 16:18

Merge branch 'main' of github.com:elastic/kibana into feature/seed-si…

6b5afc5

…gevents-env

fix: tsconfig

a171cb2

refactor(streams): improve seed script types, architecture, and synth…

53076d4

…trace usage

crespocarlos marked this pull request as ready for review April 6, 2026 21:05

crespocarlos requested review from a team as code owners April 6, 2026 21:05

crespocarlos requested a review from ruflin April 6, 2026 21:05

fix: seed insights

1c2a26c

macroscopeapp Bot reviewed Apr 8, 2026

View reviewed changes

Comment thread x-pack/platform/plugins/shared/streams/scripts/seed_sigevents_env/steps/seed_tasks.ts Outdated

crespocarlos added 2 commits April 8, 2026 10:40

fix: insights cleanup

d6eca5b

Merge branch 'main' into feature/seed-sigevents-env

d8aa8b6

Merge branch 'main' into feature/seed-sigevents-env

27c6f48

ruflin approved these changes Apr 14, 2026

View reviewed changes

Merge branch 'main' into feature/seed-sigevents-env

638a69c

macroscopeapp Bot reviewed Apr 14, 2026

View reviewed changes

Comment thread x-pack/platform/plugins/shared/streams/tsconfig.json

fix: merge

507bce5

crespocarlos merged commit d5886ed into elastic:main Apr 14, 2026
21 checks passed

kibanamachine added the v9.5.0 label Apr 14, 2026

crespocarlos deleted the feature/seed-sigevents-env branch April 14, 2026 14:24

Conversation

crespocarlos commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

macroscopeapp Bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

ruflin commented Apr 8, 2026

Uh oh!

crespocarlos commented Apr 8, 2026

Uh oh!

Uh oh!

ruflin commented Apr 8, 2026

Uh oh!

crespocarlos commented Apr 8, 2026

Uh oh!

crespocarlos commented Apr 9, 2026

Uh oh!

ruflin commented Apr 10, 2026

Uh oh!

crespocarlos commented Apr 14, 2026

Uh oh!

crespocarlos commented Apr 14, 2026

Uh oh!

elasticmachine commented Apr 14, 2026

Uh oh!

crespocarlos commented Apr 14, 2026

Uh oh!

elasticmachine commented Apr 14, 2026

Uh oh!

Uh oh!

elasticmachine commented Apr 14, 2026

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

crespocarlos commented Apr 3, 2026 •

edited

Loading

macroscopeapp Bot commented Apr 6, 2026 •

edited

Loading