Skip to content

[Security Solution] Fix race condition when Defend is installed before rules package#222200

Merged
xcrzx merged 1 commit intoelastic:mainfrom
xcrzx:fix-elastic-defend-race
Jun 3, 2025
Merged

[Security Solution] Fix race condition when Defend is installed before rules package#222200
xcrzx merged 1 commit intoelastic:mainfrom
xcrzx:fix-elastic-defend-race

Conversation

@xcrzx
Copy link
Contributor

@xcrzx xcrzx commented Jun 2, 2025 •
edited by kibanamachine
Loading

Summary

Fixes a race condition where the Elastic Defend rule is installed before the rules package becomes available, resulting in the following error:

[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension] Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id: 9a1a2dae-0b5f-4c3d-8305-a268d404c306)

Steps to Reproduce

  1. Start with a clean Kibana instance with no rules package installed.
  2. Navigate directly to the Integrations page (without visiting any Security Solution pages, which would trigger rules package bootstrapping), and install the Elastic Defend integration.
  3. Observe the Unable to find Elastic Defend rule in the prebuilt rule assets error in the Kibana logs.

This race condition appears to have existed for some time but was surfaced more clearly due to the recently added warning when the Defend rule cannot be installed.

@xcrzx xcrzx self-assigned this Jun 2, 2025
@xcrzx xcrzx added bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Jun 2, 2025
@xcrzx xcrzx marked this pull request as ready for review June 2, 2025 15:03
@xcrzx xcrzx requested a review from a team as a code owner June 2, 2025 15:03
@xcrzx xcrzx requested a review from maximpn June 2, 2025 15:03
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 2, 2025

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 2, 2025

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 2, 2025

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@xcrzx xcrzx force-pushed the fix-elastic-defend-race branch from 91f348b to d667a7f Compare June 2, 2025 15:12
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 2, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #4 / serverless search UI - feature flags Serverless Synonyms Overview Synonyms get started Page should not override existing synonyms set unless specified

Metrics [docs]

✅ unchanged

History

cc @xcrzx

@maximpn maximpn changed the title [Security Solution] Fix race condition when Defend is installed before rules pacakge [Security Solution] Fix race condition when Defend is installed before rules package Jun 2, 2025
maximpn
maximpn approved these changes Jun 2, 2025
View reviewed changes
Copy link
Contributor

@maximpn maximpn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xcrzx Thanks for fixing this race condition bug 👍

The diff looks straightforward. I tested the changes locally and it works as expected. The only side effect is longer Elastic Defend integration installation due to Security Detection Rules package (considered as an integration in Fleet) installation happening under the hood.

I've noticed Fleet logs starting integration installation like Install with state machine - Starting installation of security_detection_engine@9.0.5 from registry but there isn't a massage saying it's been successfully installed.

@xcrzx xcrzx merged commit c513625 into elastic:main Jun 3, 2025
10 checks passed
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 3, 2025

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15412835448

@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 3, 2025

💔 All backports failed

Status Branch Result
❌ 8.19 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 222200

Questions ?

Please refer to the Backport tool documentation

zacharyparikh pushed a commit to zacharyparikh/kibana that referenced this pull request Jun 4, 2025
@zacharyparikh
[Security Solution] Fix race condition when Defend is installed befor…
ddd6196 
…e rules package (elastic#222200)

## Summary

Fixes a race condition where the Elastic Defend rule is installed before
the rules package becomes available, resulting in the following error:

```
[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension] Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id: 9a1a2dae-0b5f-4c3d-8305-a268d404c306)
```

### Steps to Reproduce

1. Start with a clean Kibana instance with no rules package installed.  
2. Navigate directly to the Integrations page (without visiting any
Security Solution pages, which would trigger rules package
bootstrapping), and install the Elastic Defend integration.
3. Observe the `Unable to find Elastic Defend rule in the prebuilt rule
assets` error in the Kibana logs.

This race condition appears to have existed for some time but was
surfaced more clearly due to the recently added warning when the Defend
rule cannot be installed.
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jun 5, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 6, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

@xcrzx
Copy link
Contributor Author

xcrzx commented Jun 6, 2025 •
edited
Loading

Waiting till some other AI4SOC PRs get backported to 8.19 so this one can be merged on top of them

@elastic elastic deleted a comment from kibanamachine Jun 6, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 9, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

1 similar comment
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 10, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 11, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

1 similar comment
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 12, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

nickpeihl pushed a commit to nickpeihl/kibana that referenced this pull request Jun 12, 2025
@nickpeihl
[Security Solution] Fix race condition when Defend is installed befor…
816f5db 
…e rules package (elastic#222200)

## Summary

Fixes a race condition where the Elastic Defend rule is installed before
the rules package becomes available, resulting in the following error:

```
[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension] Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id: 9a1a2dae-0b5f-4c3d-8305-a268d404c306)
```

### Steps to Reproduce

1. Start with a clean Kibana instance with no rules package installed.  
2. Navigate directly to the Integrations page (without visiting any
Security Solution pages, which would trigger rules package
bootstrapping), and install the Elastic Defend integration.
3. Observe the `Unable to find Elastic Defend rule in the prebuilt rule
assets` error in the Kibana logs.

This race condition appears to have existed for some time but was
surfaced more clearly due to the recently added warning when the Defend
rule cannot be installed.
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 13, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

1 similar comment
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 16, 2025

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 222200 locally
cc: @xcrzx

@xcrzx xcrzx added backport:version Backport to applied version labels and removed backport missing Added to PRs automatically when the are determined to be missing a backport. backport:version Backport to applied version labels labels Jun 17, 2025
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 17, 2025

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15702950293

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jun 17, 2025
[Security Solution] Fix race condition when Defend is installed befor…
54780d7 
…e rules package (elastic#222200)

## Summary

Fixes a race condition where the Elastic Defend rule is installed before
the rules package becomes available, resulting in the following error:

```
[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension] Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id: 9a1a2dae-0b5f-4c3d-8305-a268d404c306)
```

### Steps to Reproduce

1. Start with a clean Kibana instance with no rules package installed.
2. Navigate directly to the Integrations page (without visiting any
Security Solution pages, which would trigger rules package
bootstrapping), and install the Elastic Defend integration.
3. Observe the `Unable to find Elastic Defend rule in the prebuilt rule
assets` error in the Kibana logs.

This race condition appears to have existed for some time but was
surfaced more clearly due to the recently added warning when the Defend
rule cannot be installed.

(cherry picked from commit c513625)
@kibanamachine kibanamachine mentioned this pull request Jun 17, 2025
Merged
@kibanamachine
Copy link
Contributor

kibanamachine commented Jun 17, 2025

💚 All backports created successfully

Status Branch Result
✅ 8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Jun 17, 2025
@kibanamachine
[8.19] [Security Solution] Fix race condition when Defend is installe…
07b9e73 
…d before rules package (#222200) (#224193)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Security Solution] Fix race condition when Defend is installed
before rules package
(#222200)](#222200)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Dmitrii
Shevchenko","email":"dmitrii.shevchenko@elastic.co"},"sourceCommit":{"committedDate":"2025-06-03T08:50:03Z","message":"[Security
Solution] Fix race condition when Defend is installed before rules
package (#222200)\n\n## Summary\n\nFixes a race condition where the
Elastic Defend rule is installed before\nthe rules package becomes
available, resulting in the following
error:\n\n```\n[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension]
Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id:
9a1a2dae-0b5f-4c3d-8305-a268d404c306)\n```\n\n### Steps to
Reproduce\n\n1. Start with a clean Kibana instance with no rules package
installed. \n2. Navigate directly to the Integrations page (without
visiting any\nSecurity Solution pages, which would trigger rules
package\nbootstrapping), and install the Elastic Defend integration.\n3.
Observe the `Unable to find Elastic Defend rule in the prebuilt
rule\nassets` error in the Kibana logs.\n\nThis race condition appears
to have existed for some time but was\nsurfaced more clearly due to the
recently added warning when the Defend\nrule cannot be
installed.","sha":"c513625f1181a4a47b8c308e2f1eb38f6b68830f","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:skip","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","backport:version","v9.1.0","v8.19.0"],"title":"[Security
Solution] Fix race condition when Defend is installed before rules
package","number":222200,"url":"https://github.com/elastic/kibana/pull/222200","mergeCommit":{"message":"[Security
Solution] Fix race condition when Defend is installed before rules
package (#222200)\n\n## Summary\n\nFixes a race condition where the
Elastic Defend rule is installed before\nthe rules package becomes
available, resulting in the following
error:\n\n```\n[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension]
Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id:
9a1a2dae-0b5f-4c3d-8305-a268d404c306)\n```\n\n### Steps to
Reproduce\n\n1. Start with a clean Kibana instance with no rules package
installed. \n2. Navigate directly to the Integrations page (without
visiting any\nSecurity Solution pages, which would trigger rules
package\nbootstrapping), and install the Elastic Defend integration.\n3.
Observe the `Unable to find Elastic Defend rule in the prebuilt
rule\nassets` error in the Kibana logs.\n\nThis race condition appears
to have existed for some time but was\nsurfaced more clearly due to the
recently added warning when the Defend\nrule cannot be
installed.","sha":"c513625f1181a4a47b8c308e2f1eb38f6b68830f"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/222200","number":222200,"mergeCommit":{"message":"[Security
Solution] Fix race condition when Defend is installed before rules
package (#222200)\n\n## Summary\n\nFixes a race condition where the
Elastic Defend rule is installed before\nthe rules package becomes
available, resulting in the following
error:\n\n```\n[2025-05-29T10:40:00.066-04:00][ERROR][plugins.securitySolution.endpointFleetExtension]
Unable to find Elastic Defend rule in the prebuilt rule assets (rule_id:
9a1a2dae-0b5f-4c3d-8305-a268d404c306)\n```\n\n### Steps to
Reproduce\n\n1. Start with a clean Kibana instance with no rules package
installed. \n2. Navigate directly to the Integrations page (without
visiting any\nSecurity Solution pages, which would trigger rules
package\nbootstrapping), and install the Elastic Defend integration.\n3.
Observe the `Unable to find Elastic Defend rule in the prebuilt
rule\nassets` error in the Kibana logs.\n\nThis race condition appears
to have existed for some time but was\nsurfaced more clearly due to the
recently added warning when the Defend\nrule cannot be
installed.","sha":"c513625f1181a4a47b8c308e2f1eb38f6b68830f"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dmitrii Shevchenko <dmitrii.shevchenko@elastic.co>
@xcrzx xcrzx deleted the fix-elastic-defend-race branch June 17, 2025 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maximpn maximpn maximpn approved these changes

Assignees

@xcrzx xcrzx

Labels

backport:version Backport to applied version labels bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.19.0 v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@xcrzx @elasticmachine @kibanamachine @maximpn