[Cases] Add extract observables setting to cases SO mapping by christineweng · Pull Request #236028 · elastic/kibana

christineweng · 2025-09-23T03:57:18Z

Summary

This is part 1 of introducing auto-extract observables in cases. This PR adds mapping and default value of false to a new setting called extractObservables.

Part 2 that introduces the actual feature is in #233027

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

elasticmachine · 2025-09-23T15:14:32Z

Pinging @elastic/kibana-cases (Team:Cases)

PhilippeOberti

Code review only (a single Cypress file owned by our team). LGTM for the @elastic/security-threat-hunting team

christineweng · 2025-09-23T20:53:19Z

Files by Code Owner (non-cases teams)

elastic/appex-qa

x-pack/platform/test/serverless/api_integration/services/svl_cases/api.ts

elastic/kibana-core

packages/kbn-check-saved-objects-cli/current_fields.json
packages/kbn-check-saved-objects-cli/current_mappings.json
src/core/server/integration_tests/ci_checks/saved_objects/check_registered_types.test.ts

elastic/security-detections-response

x-pack/solutions/security/test/security_solution_cypress/cypress/objects/case.ts

elastic/security-threat-hunting

x-pack/solutions/security/test/security_solution_cypress/cypress/objects/case.ts

rylnd

Detection engine change (to a mock Case object in our cypress suite) LGTM.

csr

x-pack/platform/test/serverless/api_integration/services/svl_cases/api.ts LGTM 👍

jloleysens

SO data migrations LGTM.

Part 2 that introduces the actual feature is in #233027

Just note that this data migration will need to be released separately from the feature code that depends on it (i.e., target the following serverless release). Otherwise you can start writing feature code that treats the new field as optional to unblock your progress.

jloleysens · 2025-09-24T14:33:28Z

...tform/plugins/shared/cases/server/saved_object_types/cases/model_versions/model_version_5.ts

+    {
+      type: 'mappings_addition',
+      addedMappings: {
+        settings: {
+          properties: {
+            extractObservables: {
+              type: 'boolean',
+            },
+          },
+        },
+      },
+    },
+    {
+      type: 'data_backfill',
+      backfillFn: (doc) => {
+        const settings = doc.attributes.settings;
+        return { attributes: { settings: { ...settings, extractObservables: false } } };
+      },
+    },


My apologies for not calling this out in my first pass. These new mappings will need to be added in a new model version. Model version 6. And they should not aim to be released along with model version 5.

Hey @jloleysens . Model Version 5 hasn't gone out into serverless yet. I merged that yesterday just in case.

This PR: #235290

jloleysens

Thanks for raising and confirming 5 hasn't been released @christineweng and @michaelolo24 !

You might just consider bundling them together in one change object, other than that I think this change makes sense.

jloleysens

Thanks for raising and confirming 5 hasn't been released @christineweng and @michaelolo24 !

You might just consider bundling them together in one change object, other than that I think this change makes sense.

michaelolo24 · 2025-09-24T21:13:30Z

x-pack/platform/plugins/shared/cases/server/saved_object_types/cases/versioning.test.ts

+          },
+          overrides: {
+            severity: 10, // Severity value that matches v1 schema (10, 20, 30, 40)
+            // @ts-expect-error: total_events is not defined in the attributes


@lgestc we may want to add this field to the actual attributes for the mapping per what was done for alerts here: #223992 and it's utility in the analytics indices. Fyi @janmonschke

michaelolo24 · 2025-09-24T21:15:07Z

x-pack/platform/plugins/shared/cases/common/types/domain/case/v1.test.ts

-    const query = CaseSettingsRt.decode({ syncAlerts: false, foo: 'bar' });
+  // Stripping additional attributes require a stricter rt type
+  // This test cannot be passed because we want extractObservables to be optional
+  it.skip('removes foo:bar attributes from request', () => {


spoke offline about this, but I think we can fix this in this PR or a follow up with:

rt.intersection([ rt.strict({ syncAlerts: rt.boolean, }), rt.exact( rt.partial({ extractObservables: rt.boolean, }) ), ]);

michaelolo24

Nice work! @christineweng and I were able to test the upgrade scenario via the qaf update util and the backfill worked as expect both on existing older cases in the cluster, as well as imported older cases as well. Nice work!

elasticmachine · 2025-09-25T05:23:16Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 977d914
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-236028-977d91465586
Security Deployment

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #100 / endpoint endpoint list when initially navigating to page when there is data, "after all" hook for "does not show the details flyout initially"
[job] [logs] FTR Configs #50 / Space awareness actions POST /agents/actions/{actionId}/cancel should return 404 if the action is in a different space

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`cases`	137.3KB	137.4KB	+69.0B

History

💚 Build #343383 succeeded 40ba7d2
💔 Build #343295 failed b90a54b
💔 Build #342872 failed 3afd7f7
💔 Build #342707 failed 225c562

cc @christineweng

…236028) ## Summary This is part 1 of introducing auto-extract observables in cases. This PR adds mapping and default value of false to a new setting called `extractObservables`. Part 2 that introduces the actual feature is in elastic#233027 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...

## Summary This is part 1 of introducing auto-extract observables in cases. This PR adds mapping and default value of false to a new setting called `extractObservables`. Part 2 that introduces the actual feature is in #233027 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...

## Summary This is part 2 of the auto-extract observables work. Dependency: #236028 to be merged first. This PR adds functionality to auto-extract case observables when attaching alerts to a case via alert flyout. This is enabled in security solution cases only. The toggle should not appear in Observability and Stack cases. With platinum+ license, users in security can now: - See a toggle to auto-extract observables when attaching alert to a case - Configure the toggle in case templates - Access the toggle in case view - Access the default observables type to ecs mapping via the gear icon This feature is disabled by default, to enable it, revert this commit efc88aa Fixes #234513 https://github.com/user-attachments/assets/d74eb7bc-feb7-4208-8f4c-a99495563609 #### Not in scope in this PR - Adding toggle in alerts table (bulk actions and row actions) - Observable actions in activity (X observables were extracted) - Currently the case setting activity include both sync alert and auto-extract observables, they can be split into their own user actions - When user toggle it on in case view, perform a look back and extract all the net new observables ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. --------- Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co> Co-authored-by: lgestc <11671118+lgestc@users.noreply.github.com>

…236028) ## Summary This is part 1 of introducing auto-extract observables in cases. This PR adds mapping and default value of false to a new setting called `extractObservables`. Part 2 that introduces the actual feature is in elastic#233027 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ...

…ic#233027) ## Summary This is part 2 of the auto-extract observables work. Dependency: elastic#236028 to be merged first. This PR adds functionality to auto-extract case observables when attaching alerts to a case via alert flyout. This is enabled in security solution cases only. The toggle should not appear in Observability and Stack cases. With platinum+ license, users in security can now: - See a toggle to auto-extract observables when attaching alert to a case - Configure the toggle in case templates - Access the toggle in case view - Access the default observables type to ecs mapping via the gear icon This feature is disabled by default, to enable it, revert this commit efc88aa Fixes elastic#234513 https://github.com/user-attachments/assets/d74eb7bc-feb7-4208-8f4c-a99495563609 #### Not in scope in this PR - Adding toggle in alerts table (bulk actions and row actions) - Observable actions in activity (X observables were extracted) - Currently the case setting activity include both sync alert and auto-extract observables, they can be split into their own user actions - When user toggle it on in case view, perform a look back and extract all the net new observables ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. --------- Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co> Co-authored-by: lgestc <11671118+lgestc@users.noreply.github.com>

christineweng force-pushed the cases-extract-observables-mapping branch 3 times, most recently from aea46d5 to 225c562 Compare September 23, 2025 15:10

christineweng self-assigned this Sep 23, 2025

christineweng added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Cases Security Solution Cases team v9.2.0 labels Sep 23, 2025

christineweng marked this pull request as ready for review September 23, 2025 15:14

christineweng requested review from a team as code owners September 23, 2025 15:14

PhilippeOberti approved these changes Sep 23, 2025

View reviewed changes

christineweng force-pushed the cases-extract-observables-mapping branch from 225c562 to a6c33c0 Compare September 23, 2025 20:46

christineweng removed request for a team September 23, 2025 20:46

christineweng force-pushed the cases-extract-observables-mapping branch from a6c33c0 to 3afd7f7 Compare September 23, 2025 20:51

rylnd approved these changes Sep 23, 2025

View reviewed changes

csr approved these changes Sep 24, 2025

View reviewed changes

jloleysens approved these changes Sep 24, 2025

View reviewed changes

jloleysens requested changes Sep 24, 2025

View reviewed changes

jloleysens approved these changes Sep 24, 2025

View reviewed changes

christineweng mentioned this pull request Sep 24, 2025

[Cases] Auto-extract cases observables in add to case workflow #233027

Merged

6 tasks

christineweng force-pushed the cases-extract-observables-mapping branch from 543dd3c to bfad673 Compare September 24, 2025 17:22

christineweng added the ci:project-deploy-security Create a Security Serverless Project label Sep 24, 2025

add extract observables setting to SO

40ba7d2

christineweng force-pushed the cases-extract-observables-mapping branch from bfad673 to 40ba7d2 Compare September 24, 2025 17:50

michaelolo24 reviewed Sep 24, 2025

View reviewed changes

michaelolo24 approved these changes Sep 24, 2025

View reviewed changes

fix test

977d914

christineweng merged commit f8bec08 into elastic:main Sep 25, 2025
12 checks passed

christineweng removed the ci:project-deploy-security Create a Security Serverless Project label Sep 25, 2025

Conversation

christineweng commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Identify risks

Uh oh!

elasticmachine commented Sep 23, 2025

Uh oh!

PhilippeOberti left a comment

Choose a reason for hiding this comment

Uh oh!

christineweng commented Sep 23, 2025

Files by Code Owner (non-cases teams)

elastic/appex-qa

elastic/kibana-core

elastic/security-detections-response

elastic/security-threat-hunting

Uh oh!

rylnd left a comment

Choose a reason for hiding this comment

Uh oh!

csr left a comment

Choose a reason for hiding this comment

Uh oh!

jloleysens left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

michaelolo24 Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens left a comment

Choose a reason for hiding this comment

Uh oh!

jloleysens left a comment

Choose a reason for hiding this comment

Uh oh!

michaelolo24 Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelolo24 Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

michaelolo24 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Page load bundle

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

christineweng commented Sep 23, 2025 •

edited

Loading

jloleysens left a comment •

edited

Loading

michaelolo24 Sep 24, 2025 •

edited

Loading

michaelolo24 Sep 24, 2025 •

edited

Loading

elasticmachine commented Sep 25, 2025 •

edited

Loading