[Security Solution] Prebuilt rule deprecation workflow automated tests by dplumlee · Pull Request #263662 · elastic/kibana

dplumlee · 2026-04-16T04:40:35Z

Summary

Adds unit tests, FTR integration tests, and cypress e2e tests for the prebuilt rule deprecation workflow feature in accordance to the test plan

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed

macroscopeapp · 2026-04-16T04:42:39Z

Catch flakiness early (recommended)

Recommended before merge: run the flaky test runner against this PR to catch flakiness early.

Trigger a run with the Flaky Test Runner UI or post this comment on the PR:

/flaky ftrConfig:x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/ess_basic_license.config.ts:30
/flaky ftrConfig:x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/serverless_essentials_tier.config.ts:30
/flaky ftrConfig:x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/ess.config.ts:30
/flaky ftrConfig:x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/serverless.config.ts:30

This check is experimental. Share your feedback in the #appex-qa channel.

^{Posted via Macroscope — Flaky Test Runner nudge}

kibanamachine · 2026-04-16T19:19:39Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11680

[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/serverless_essentials_tier.config.ts: 25/25 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/ess_basic_license.config.ts: 25/25 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/ess.config.ts: 25/25 tests passed.
[✅] x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/serverless.config.ts: 25/25 tests passed.

see run history

elasticmachine · 2026-04-16T21:25:12Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine · 2026-04-16T21:25:13Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2026-04-16T21:25:15Z

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

nikitaindik · 2026-04-22T10:22:04Z

+      });
+    });
+
+    describe('With ids filter', () => {


I noticed a small inconsistency with the test plan.

Given a non-existent rule SO id When the user requests the deprecation review filtered to the non-existent id Then the response contains an empty rules array

This would actually return a 400 with

{"message":"No rules found for bulk get","status_code":400}

You can repro by using "Copy as Fetch" in Network tab on a real request, then updating the id in the request.

Yeah I had meant to delete this, the behavior in the test plan is wrong. I've switched it over to the 400 error instead and added a specific test that lines up with the pattern we have in the endpoint

nikitaindik · 2026-04-22T10:32:40Z

For Cypress tests, I think it'd be a good idea to assert that success toasts appear after delete / duplicate+delete. This would give us reasonable confidence that operations actually succeeded.

nikitaindik · 2026-04-22T10:38:00Z

Noticed that tests for count in the bootstrap endpoint are missing. Do we want to add them?

nikitaindik · 2026-04-22T11:02:45Z

+        });
+      });
+
+      it('deletes a deprecated rule from its details page and navigates back to the rules list', () => {


nit: If delete fails, we still navigate to the rules list. Should stay on the same page, imo.

nikitaindik · 2026-04-22T12:21:31Z

Hey, @dplumlee! I reviewed, left a few suggestions / questions.

dplumlee · 2026-04-22T16:04:42Z

Noticed that tests for count in the bootstrap endpoint are missing. Do we want to add them?

Added one in 00c7fbf

macroscopeapp · 2026-04-22T16:26:29Z

Follow-up: catch flakiness early for new cypress tests

Follow-up to the previous flaky test runner nudge, covering the new cypress specs added since then.

Trigger a run with the 一laky Test Runner UI or post this comment on the PR:

/flaky ftrConfig:x-pack/solutions/security/test/security_solution_cypress/cli_config.ts:30

Covers the two new ESS cypress specs under cypress/e2e/detection_response/rule_management/prebuilt_rules/deprecation/.

^{Share feedback in the #appex-qa channel.}

^{Posted via Macroscope — Flaky Test Runner nudge}

nikitaindik

Thanks for addressing my feedback, @dplumlee! Coverage LGTM now.

kibanamachine · 2026-04-22T18:56:23Z

Starting backport for target branches: 9.4

https://github.com/elastic/kibana/actions/runs/24796786997

elasticmachine · 2026-04-22T18:58:07Z

💚 Build Succeeded

Buildkite Build
Commit: 359043d

Metrics [docs]

✅ unchanged

History

💔 Build #432324 failed 00c7fbf
💛 Build #429147 was flaky e88a208
💔 Build #428972 failed e3140ee
💔 Build #428606 failed c787b44

cc @dplumlee

kibanamachine · 2026-04-22T19:08:02Z

💚 All backports created successfully

Status	Branch	Result
✅	9.4

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…d tests (#263662) (#265134) # Backport This will backport the following commits from `main` to `9.4`: - [[Security Solution] Prebuilt rule deprecation workflow automated tests (#263662)](#263662)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Davis Plumlee <56367316+dplumlee@users.noreply.github.com>

elastic#263662) ## Summary Adds unit tests, FTR integration tests, and cypress e2e tests for the [prebuilt rule deprecation workflow feature](elastic#259673) in accordance to the [test plan](elastic#259855) ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed

adds tests from test plan

c787b44

dplumlee self-assigned this Apr 16, 2026

dplumlee added 2 commits April 16, 2026 10:10

fix types

e3140ee

fixes test

e88a208

dplumlee marked this pull request as ready for review April 16, 2026 21:25

dplumlee requested review from a team as code owners April 16, 2026 21:25

dplumlee requested review from dhurley14 and sdesalas April 16, 2026 21:25

dhurley14 approved these changes Apr 17, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/main' into deprecated-rules-tests

bcb976c

nikitaindik requested review from nikitaindik and removed request for sdesalas April 21, 2026 09:49

nikitaindik reviewed Apr 22, 2026

View reviewed changes

dplumlee added 2 commits April 22, 2026 10:10

Merge remote-tracking branch 'upstream/main' into deprecated-rules-tests

f36cbc0

addresses comments

00c7fbf

dplumlee requested a review from nikitaindik April 22, 2026 16:04

nikitaindik approved these changes Apr 22, 2026

View reviewed changes

dplumlee enabled auto-merge (squash) April 22, 2026 17:16

Merge branch 'main' into deprecated-rules-tests

359043d

dplumlee merged commit 50dec39 into elastic:main Apr 22, 2026
17 checks passed

kibanamachine mentioned this pull request Apr 22, 2026

[9.4] [Security Solution] Prebuilt rule deprecation workflow automated tests (#263662) #265134

Merged

dplumlee deleted the deprecated-rules-tests branch April 22, 2026 19:25

Conversation

dplumlee commented Apr 16, 2026 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

macroscopeapp Bot commented Apr 16, 2026

Catch flakiness early (recommended)

Uh oh!

kibanamachine commented Apr 16, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11680

Uh oh!

elasticmachine commented Apr 16, 2026

Uh oh!

elasticmachine commented Apr 16, 2026

Uh oh!

elasticmachine commented Apr 16, 2026

Uh oh!

nikitaindik Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dplumlee Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

nikitaindik Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

nikitaindik commented Apr 22, 2026

Uh oh!

nikitaindik Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

nikitaindik commented Apr 22, 2026

Uh oh!

dplumlee commented Apr 22, 2026

Uh oh!

macroscopeapp Bot commented Apr 22, 2026

Follow-up: catch flakiness early for new cypress tests

Uh oh!

nikitaindik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Apr 22, 2026

Uh oh!

elasticmachine commented Apr 22, 2026

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

kibanamachine commented Apr 22, 2026

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dplumlee commented Apr 16, 2026 •

edited by kibanamachine

Loading

nikitaindik Apr 22, 2026 •

edited

Loading