[EDR Workflows] Fix metadata transform race conditions and unskip flaky endpoint integration tests by szwarckonrad · Pull Request #261180 · elastic/kibana

szwarckonrad · 2026-04-03T19:15:16Z

Unskips two flaky endpoint integration FTR tests and fixes the underlying issues that caused them to fail.

Root causes:

startMetadataTransforms() failed with resource_not_found_exception or TimeoutError because transforms weren't created yet or ES was slow on CI
Test cleanup failed with 400 when deleting agent policies that Fleet still considered active
Both tests used loadEndpointData() with full transforms/alerts/response-actions setup when they didn't need it, wasting timeout budget

Changes:

transforms.ts — shared infrastructure improvements:

Added waitForTransformsToBeCreated() that polls for transforms to exist before starting them (replaces one-shot lookup that raced with Fleet install)
Added startTransformWithRetry() that retries on 404 and TimeoutError with backoff
Both functions benefit all tests using startMetadataTransforms, not just the two being unskipped

index_fleet_endpoint_policy.ts:

Added force: true to agent policy deletion during test cleanup to avoid race with Fleet state

artifact_entries_list.ts (Closes #249124):

Unskipped
Replaced loadEndpointData() with a single event doc indexed into logs-endpoint.events.process-default to seed field mappings for the Event Filters autocomplete — the only data this test actually needs from ES

trusted_apps_list.ts (Closes #251545):

Unskipped
Removed loadEndpointData() entirely — this test only does UI CRUD with hardcoded values, no ES field queries

Closes #249124
Closes #251545

…gration tests

kibanamachine · 2026-04-03T20:15:21Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11373

[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 0/25 tests passed.
[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/serverless.integrations.config.ts: 24/25 tests passed.

see run history

…eet state

kibanamachine · 2026-04-03T21:27:28Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11375

[✅] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 1/1 tests passed.
[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/serverless.integrations.config.ts: 47/50 tests passed.

see run history

…eout budget

kibanamachine · 2026-04-03T23:57:34Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11377

[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 48/50 tests passed.
[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/serverless.integrations.config.ts: 49/50 tests passed.

see run history

kibanamachine · 2026-04-04T01:04:18Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11379

[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 46/50 tests passed.
[❌] x-pack/solutions/security/test/security_solution_endpoint/configs/serverless.integrations.config.ts: 48/50 tests passed.

see run history

…n artifact_entries_list

… for Event Filters autocomplete

kibanamachine · 2026-04-06T20:32:49Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11398

[✅] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 2/2 tests passed.

see run history

kibanamachine · 2026-04-06T21:56:41Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11399

[✅] x-pack/solutions/security/test/security_solution_endpoint/configs/integrations.config.ts: 50/50 tests passed.
[✅] x-pack/solutions/security/test/security_solution_endpoint/configs/serverless.integrations.config.ts: 50/50 tests passed.

see run history

elasticmachine · 2026-04-06T22:34:26Z

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

elasticmachine · 2026-04-06T23:49:51Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: dcb4c90

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #214 / discover/cascade layout grouping data fetching does not refetch when returning to a previously expanded group
[job] [logs] Scout: [ platform / alerting_v2 ] plugin / local-serverless-search - Discover Alerts menu with alerting v2 - should show Alerts menu with v2 ES|QL rule row and New badge
[job] [logs] Scout: [ platform / alerting_v2 ] plugin / local-serverless-security_complete - Discover Alerts menu with alerting v2 - should show Alerts menu with v2 ES|QL rule row and New badge

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`securitySolution`	748	749	+1

Total ESLint disabled count

id	before	after	diff
`securitySolution`	853	854	+1

History

💔 Build #422128 failed 9846f4d

cc @szwarckonrad

joeypoon

🙆‍♂️

…#261845) The artifacts regrouping PR (#257001) reorganized all endpoint artifacts into a unified page with tabs. This changed the page structure so that `header-page-title` now belongs to the unified "Artifacts" page and is always present, regardless of whether individual tabs have entries. Previously, `trusted_apps_list.ts` was skipped (flaky). Our PR #261180 unskipped it after fixing the underlying flakiness — but by that point the regrouping had already landed, making the `header-page-title` assertions invalid. The test immediately started failing on merge. `artifact_entries_list.ts` has the same `header-page-title` assertions for all artifact types (trusted apps, event filters, blocklist, host isolation exceptions) — these would fail for the same reason once hit. **Fix:** Replace `missingOrFail('header-page-title')` with `existOrFail('*-emptyState')` in both test files. The empty state test subject is the correct indicator that a tab has no entries. Closes #261827 Closes #261829 Closes #261833 Closes #261849 Closes #261850

Fix metadata transform race conditions and unskip flaky endpoint inte…

ca7d20f

…gration tests

szwarckonrad added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution labels Apr 3, 2026

szwarckonrad self-assigned this Apr 3, 2026

szwarckonrad added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution labels Apr 3, 2026

Flaky test runner config, to be reversed

9846f4d

Force delete agent policies during test cleanup to avoid race with Fl…

402d59e

…eet state

szwarckonrad added 2 commits April 3, 2026 16:23

Add retry with backoff for startTransform to handle CI timeouts

150ad3c

Move transform wait into startMetadataTransforms to avoid wasting tim…

e87c3ff

…eout budget

szwarckonrad and others added 4 commits April 6, 2026 09:26

Merge branch 'main' into unskip-flaky-endpoint-integration-tests

dfb8977

Remove unnecessary loadEndpointData from artifact and trusted apps tests

9fab4dc

Remove loadEndpointData from trusted_apps_list, use minimal options i…

9b8cf4f

…n artifact_entries_list

Replace loadEndpointData with single event doc to seed field mappings…

35c85ae

… for Event Filters autocomplete

reenable all test files

dcb4c90

szwarckonrad marked this pull request as ready for review April 6, 2026 22:34

szwarckonrad requested a review from a team as a code owner April 6, 2026 22:34

szwarckonrad requested review from joeypoon and pzl April 6, 2026 22:34

joeypoon approved these changes Apr 7, 2026

View reviewed changes

szwarckonrad merged commit f21bf0e into elastic:main Apr 7, 2026
21 checks passed

kibanamachine added the v9.4.0 label Apr 7, 2026

szwarckonrad mentioned this pull request Apr 7, 2026

[EDR Workflows] Fix artifact FTR tests broken by artifacts regrouping #261845

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EDR Workflows] Fix metadata transform race conditions and unskip flaky endpoint integration tests#261180

[EDR Workflows] Fix metadata transform race conditions and unskip flaky endpoint integration tests#261180
szwarckonrad merged 10 commits intoelastic:mainfrom
szwarckonrad:unskip-flaky-endpoint-integration-tests

szwarckonrad commented Apr 3, 2026 •

edited

Loading

Uh oh!

kibanamachine commented Apr 3, 2026

Uh oh!

kibanamachine commented Apr 3, 2026

Uh oh!

kibanamachine commented Apr 3, 2026

Uh oh!

kibanamachine commented Apr 4, 2026

Uh oh!

kibanamachine commented Apr 6, 2026

Uh oh!

kibanamachine commented Apr 6, 2026

Uh oh!

elasticmachine commented Apr 6, 2026

Uh oh!

elasticmachine commented Apr 6, 2026

ESLint disabled line counts

Total ESLint disabled count

Uh oh!

joeypoon left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

szwarckonrad commented Apr 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kibanamachine commented Apr 3, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11373

Uh oh!

kibanamachine commented Apr 3, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11375

Uh oh!

kibanamachine commented Apr 3, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11377

Uh oh!

kibanamachine commented Apr 4, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#11379

Uh oh!

kibanamachine commented Apr 6, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11398

Uh oh!

kibanamachine commented Apr 6, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#11399

Uh oh!

elasticmachine commented Apr 6, 2026

Uh oh!

elasticmachine commented Apr 6, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

joeypoon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

szwarckonrad commented Apr 3, 2026 •

edited

Loading