Fix Failing test: Service map embeddable by sbelastic · Pull Request #267540 · elastic/kibana

sbelastic · 2026-05-04T15:22:57Z

Closes #265639

Summary

Fix Service map embeddable
Changed things in apex QA files to give us the option to use custom timeouts, a number of the crashes was because the current hardcoded timeouts aren't enough to pass in the CI (examples: Comment 1, Comment 2, Comment 3)

Ran against ECH environment

How to run

Server

node scripts/scout.js start-server --arch stateful --domain classic

Client

npx playwright test --project local --ui --config x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts

On ECH

npx playwright test --project=ech --ui --config x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts

Checklist

Flaky Test Runner was used on any tests changed (last successful run)

macroscopeapp · 2026-05-04T15:24:06Z

Catch flakiness early (recommended): run the flaky test runner against this PR before merging.

This PR unskips a previously-failing test suite (service_map_embeddable.spec.ts, ref #265639) and adds new timeout overrides and async interactions, so 30× runs help validate stability.

Trigger a run with the Flaky Test Runner UI or post this comment on the PR:

/flaky scoutConfig:x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts:30

^{Share feedback in the #appex-qa channel.}

^{Posted via Macroscope — Flaky Test Runner nudge}

kibanamachine · 2026-05-04T19:34:15Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#12093

[✅] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch stateful --domain classic): 50/50 tests passed.
[✅] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch serverless --domain observability_complete): 50/50 tests passed.

see run history

infra-vault-gh-plugin-prod · 2026-05-05T07:54:41Z

Pinging @elastic/obs-presentation-team (Team:obs-presentation)

macroscopeapp · 2026-05-05T07:55:36Z

Approvability

Verdict: Needs human review

Unresolved review comments raise substantive concerns: one questions whether the fix addresses the actual root cause and requests verification on ECH deployment, another identifies a TypeScript type bug where useFill is missing from the selectSingleOption options type. Additionally, the author does not own any of the modified files.

No code changes detected at 310dea3. Prior analysis still applies.

^{You can customize Macroscope's approvability policy. Learn more.}

dmlemeshko · 2026-05-05T08:12:31Z

    await this.page.gotoApp('dashboards', { hash: '/create' });
-    await expect(this.addTopNavButton).toBeVisible({ timeout: 20_000 });
+    await expect(this.addTopNavButton).toBeVisible({ timeout: options?.timeout ?? 20_000 });
  }


I believe this is not the main issue with the test:

Could be that in Cloud typing Service name causes warning and affects the next steps?

I believe running flaky-test-runner against Kibana local servers doesn't really help to verify the fix. Please run the test against ECH deployment to confirm the issue is fixed.

Could be that in Cloud typing Service name causes warning and affects the next steps?

Yes, this would break things.

A quick overview of the changes is:

Increase timeout from 20 seconds to 45 seconds because, running locally against an ECH environment over 30 times, one of them it got to the 20 second timeout

Added an explicit closing of a popup that caused some flakiness

[Newest change] Added an option to fill the search input instead of pressing each key (reduced the flakiness A LOT)

Not directly related to the test, more code related, but added the option for other parts of the same files to have a custom timeout or to use fill instead of type for future tests.

Note: Added photo of ECH test passing (did a commit afterwards because the date input is not the same locally vs ECH, everything else is the same and should pass)

dmlemeshko

Keeping approve back before ECH run confirmation since we might increase timeout without good reason (45 sec even for ECH is too much)

sbelastic · 2026-05-05T08:48:11Z

/oblt-deploy

kibanamachine · 2026-05-05T12:10:51Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#12103

[❌] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch stateful --domain classic): 0/50 tests passed.
[❌] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch serverless --domain observability_complete): 0/50 tests passed.

see run history

kibanamachine · 2026-05-05T13:10:09Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#12105

[✅] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch stateful --domain classic): 50/50 tests passed.
[✅] x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts (--arch serverless --domain observability_complete): 50/50 tests passed.

see run history

…map-embeddable-test

jennypavlova

LGTM, thanks for fixing the test!

kibanamachine · 2026-05-06T13:05:20Z

💚 Build Succeeded

Buildkite Build
Commit: 62c9efa

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/scout`	226	227	+1
`@kbn/scout-oblt`	255	256	+1
`@kbn/scout-search`	226	227	+1
`@kbn/scout-security`	246	247	+1
total			+4

Unknown metric groups

API count

id	before	after	diff
`@kbn/scout`	698	699	+1
`@kbn/scout-oblt`	706	707	+1
`@kbn/scout-search`	666	667	+1
`@kbn/scout-security`	686	687	+1
total			+4

History

💔 Build #438657 failed 29efc6c
💛 Build #438567 was flaky f568d63
💔 Build #438457 failed 310dea3
💔 Build #438364 failed b24801f
💔 Build #438306 failed b1bb6d0
💔 Build #438285 failed 53060d3

cc @sbelastic

Closes elastic#265639 ## Summary - Fix Service map embeddable - Changed things in apex QA files to give us the option to use custom timeouts, a number of the crashes was because the current hardcoded timeouts aren't enough to pass in the CI (examples: [Comment 1](elastic#265639 (comment)), [Comment 2](elastic#265639 (comment)), [Comment 3](elastic#265639 (comment))) ### Ran against ECH environment <img width="1918" height="1134" alt="image" src="https://github.com/user-attachments/assets/de731bbf-2825-420f-bd29-6bb0baae5887" /> ### How to run #### Server `node scripts/scout.js start-server --arch stateful --domain classic` #### Client `npx playwright test --project local --ui --config x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts` #### On ECH `npx playwright test --project=ech --ui --config x-pack/solutions/observability/plugins/apm/test/scout/ui/parallel.playwright.config.ts` ### Checklist - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed [(last successful run)](elastic#267540 (comment))

Fix failing test

740bb16

sbelastic self-assigned this May 4, 2026

sbelastic added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting workstream:apm presentation labels May 4, 2026

sbelastic marked this pull request as ready for review May 5, 2026 07:50

sbelastic requested review from a team as code owners May 5, 2026 07:50

dmlemeshko approved these changes May 5, 2026

View reviewed changes

botelastic Bot added the Team:obs-presentation Focus: APM UI, Infra UI, Hosts UI, Universal Profiling, Obs Overview and left Navigation label May 5, 2026

dmlemeshko reviewed May 5, 2026

View reviewed changes

dmlemeshko requested changes May 5, 2026

View reviewed changes

sbelastic added 3 commits May 5, 2026 11:27

Fix type in ECH

61ee15d

Fix local errors

ca01ad0

Merge branch 'main' into 265639-fix-service-map-embeddable-test

6f38986

sbelastic requested a review from dmlemeshko May 5, 2026 13:32

Merge remote-tracking branch 'upstream/main' into 265639-fix-service-…

53060d3

…map-embeddable-test

macroscopeapp Bot reviewed May 5, 2026

View reviewed changes

Comment thread src/platform/packages/shared/kbn-scout/src/playwright/eui_components/combo_box.ts

sbelastic added 4 commits May 5, 2026 17:05

Fix missing option

b1bb6d0

Merge branch 'main' into 265639-fix-service-map-embeddable-test

b24801f

Merge branch 'main' into 265639-fix-service-map-embeddable-test

310dea3

Fix type

528ef47

Merge branch 'main' into 265639-fix-service-map-embeddable-test

f568d63

jennypavlova approved these changes May 6, 2026

View reviewed changes

Merge branch 'main' into 265639-fix-service-map-embeddable-test

29efc6c

dmlemeshko approved these changes May 6, 2026

View reviewed changes

Merge branch 'main' into 265639-fix-service-map-embeddable-test

62c9efa

sbelastic enabled auto-merge (squash) May 6, 2026 13:00

sbelastic merged commit 2dbea04 into elastic:main May 6, 2026
30 checks passed

kibanamachine added the v9.5.0 label May 6, 2026

github-actions Bot mentioned this pull request May 8, 2026

Failing test: Service map embeddable - adds Service map panel with service name, environment and KQL filter #265639

Open

Conversation

sbelastic commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Ran against ECH environment

How to run

Server

Client

On ECH

Checklist

Uh oh!

macroscopeapp Bot commented May 4, 2026

Uh oh!

kibanamachine commented May 4, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#12093

Uh oh!

infra-vault-gh-plugin-prod Bot commented May 5, 2026

Uh oh!

macroscopeapp Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

dmlemeshko May 5, 2026

Choose a reason for hiding this comment

Uh oh!

sbelastic May 5, 2026

Choose a reason for hiding this comment

Uh oh!

dmlemeshko left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sbelastic commented May 5, 2026

Uh oh!

kibanamachine commented May 5, 2026

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#12103

Uh oh!

kibanamachine commented May 5, 2026

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#12105

Uh oh!

Uh oh!

jennypavlova left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented May 6, 2026

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sbelastic commented May 4, 2026 •

edited

Loading

macroscopeapp Bot commented May 5, 2026 •

edited

Loading

dmlemeshko left a comment •

edited

Loading