Increase ss prebuilt rules cypress timeout #230969
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
delanni
added
Team:Operations
Contributor
|
Pinging @elastic/kibana-operations (Team:Operations) |
…ion Cypress Tests timeout to 80m across the board
delanni
force-pushed
the
increase-ss-prebuilt-rules-cypress-timeout
branch
from
7194760 to
cf31012
Compare
Contributor
Author
|
/ci |
Contributor
💚 Build Succeeded
Metrics [docs]
History
|
mistic
approved these changes
Aug 7, 2025
Contributor
@delanni Thanks for the ping, @nikitaindik from our team is currently working on splitting the prebuilt rules cypress tests into multiple smaller groups. Can you please remind me what the Also, I'd suggest to backport this PR to the |
nikitaindik
added a commit
that referenced
this pull request
Aug 12, 2025
**Part of epic: #229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit.
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this pull request
Aug 12, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit. (cherry picked from commit 40b7b42) # Conflicts: # .buildkite/pipelines/chrome_forward_testing.yml # .buildkite/pipelines/es_serverless/verify_es_serverless_image.yml # .buildkite/pipelines/on_merge.yml # .buildkite/pipelines/pointer_compression.yml # .buildkite/pipelines/pull_request/security_solution/rule_management.yml # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/package.json # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/revert_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this pull request
Aug 12, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit. (cherry picked from commit 40b7b42) # Conflicts: # .buildkite/pipelines/chrome_forward_testing.yml # .buildkite/pipelines/es_serverless/verify_es_serverless_image.yml # .buildkite/pipelines/on_merge.yml # .buildkite/pipelines/pointer_compression.yml # .buildkite/pipelines/pull_request/security_solution/rule_management.yml # x-pack/solutions/security/test/security_solution_cypress/cypress/README.md # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/package.json # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this pull request
Aug 12, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit. (cherry picked from commit 40b7b42) # Conflicts: # .buildkite/pipelines/chrome_forward_testing.yml # .buildkite/pipelines/es_serverless/verify_es_serverless_image.yml # .buildkite/pipelines/on_merge.yml # .buildkite/pipelines/pointer_compression.yml # .buildkite/pipelines/pull_request/security_solution/rule_management.yml # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/package.json # x-pack/test/security_solution_cypress/cypress/README.md # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/revert_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this pull request
Aug 12, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit. (cherry picked from commit 40b7b42) # Conflicts: # .buildkite/pipelines/chrome_forward_testing.yml # .buildkite/pipelines/es_serverless/verify_es_serverless_image.yml # .buildkite/pipelines/on_merge.yml # .buildkite/pipelines/pointer_compression.yml # .buildkite/pipelines/pull_request/security_solution/rule_management.yml # x-pack/solutions/security/test/security_solution_cypress/cypress/README.md # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/package.json # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts
nikitaindik
added a commit
to nikitaindik/kibana
that referenced
this pull request
Aug 12, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit. (cherry picked from commit 40b7b42) # Conflicts: # .buildkite/pipelines/chrome_forward_testing.yml # .buildkite/pipelines/es_serverless/verify_es_serverless_image.yml # .buildkite/pipelines/on_merge.yml # .buildkite/pipelines/pointer_compression.yml # .buildkite/pipelines/pull_request/security_solution/rule_management.yml # x-pack/solutions/security/test/security_solution_cypress/cypress/README.md # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/revert_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/customization/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_update_authorization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_via_fleet.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/installation/install_workflow.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/export_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/import_prebuilt_rule.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management/management.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/solutions/security/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts # x-pack/solutions/security/test/security_solution_cypress/package.json # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/export_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/import_prebuilt_rule.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_update_authorization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_via_fleet.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/install_workflow.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/management.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/rule_customization.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_error_handling.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_notifications.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_with_preview_basic_license.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview.cy.ts # x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management/prebuilt_rules/upgrade_without_preview_basic_license.cy.ts
73 tasks
fkanout
pushed a commit
to fkanout/kibana
that referenced
this pull request
Aug 14, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit.
NicholasPeretti
pushed a commit
to NicholasPeretti/kibana
that referenced
this pull request
Aug 18, 2025
## Summary These tests have been dancing around the 60m mark, now they're often timing out. (e.g.: https://buildkite.com/elastic/kibana-on-merge/builds/74840) - this PR raises parallelism to 2x to avoid timing out. @elastic/security-detection-rule-management - can you take a look if the 60m timeout is reasonable? If so, please optimize the run time, if not, let's keep this new timeout.
NicholasPeretti
pushed a commit
to NicholasPeretti/kibana
that referenced
this pull request
Aug 18, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit.
6 tasks
qn895
pushed a commit
to qn895/kibana
that referenced
this pull request
Aug 26, 2025
## Summary These tests have been dancing around the 60m mark, now they're often timing out. (e.g.: https://buildkite.com/elastic/kibana-on-merge/builds/74840) - this PR raises parallelism to 2x to avoid timing out. @elastic/security-detection-rule-management - can you take a look if the 60m timeout is reasonable? If so, please optimize the run time, if not, let's keep this new timeout.
qn895
pushed a commit
to qn895/kibana
that referenced
this pull request
Aug 26, 2025
**Part of epic: elastic#229688 ## Summary We have recently observed that some prebuilt detection rules in Cypress tests run for over 60 minutes in CI pipelines. This leads to pipeline timeouts. As a temporary measure `parallelism` was increased to 2 [recently](elastic#230969). This PR divides the Cypress tests for prebuilt detection rules into four separate groups to avoid hitting the 60m limit. ## Changes - Divided prebuilt rules directory `prebuilt_rules` into 4 subdirs: - `installation` - `upgrade` - `customization` - `management` (enabling/disabling, deletion, import and export of rules) - Added `yarn` commands and `bash` scripts to run tests from each subdirectory - Registered new groups in BuildKite pipelines (with `parallelism: 1`): - `pull_request/security_solution/rule_management.yml` - `on_merge.yml` - `chrome_forward_testing.yml` - `pointer_compression.yml` - `verify_es_serverless_image.yml` (Serverless tests only) - Registered the new scripts in the Flaky Test Runner ## Running times in pull request pipeline Link to this PR's run in pull request pipeline: https://buildkite.com/elastic/kibana-pull-request/builds/327944 **Non-Serverless groups:** - 13 min: Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 13 min: Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 10 min: Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 31 min: Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests **Serverless groups:** - 17 min: Serverless Rule Management - Prebuilt Rules Customization - Security Solution Cypress Tests - 16 min: Serverless Rule Management - Prebuilt Rules Installation - Security Solution Cypress Tests - 11 min: Serverless Rule Management - Prebuilt Rules Management - Security Solution Cypress Tests - 27 min: Serverless Rule Management - Prebuilt Rules Upgrade - Security Solution Cypress Tests Each group's running time is significantly below the 60-minute limit.
43 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
These tests have been dancing around the 60m mark, now they're often timing out. (e.g.: https://buildkite.com/elastic/kibana-on-merge/builds/74840) - this PR raises parallelism to 2x to avoid timing out.
@elastic/security-detection-rule-management - can you take a look if the 60m timeout is reasonable? If so, please optimize the run time, if not, let's keep this new timeout.