unskip add panel flyout tests

[nreese]

Closes #259443 and #259576

Problem

#267325 broke tests dashboard_panel_listing.spec.ts and dashboard_panel_listing_obs_group.spec.ts. At the time #267325 merged, there was a bug in CI where CI did not run dashboard tests when code changes were not in dependency chain yet code changes affected registries and had a direct impact on dashboard runtime behavior.

The reason the tests broke is that #267325 changed the order in which "Add panel" actions are registered. Before #267325, ml actions where registered async - so they got registered last. After #267325, ml actions where registered sync, so they go registered before some other actions.

Registration order mattered because there was a bug in dashboard_panel_listing.spec.ts and dashboard_panel_listing_obs_group.spec.ts tests. The logic to extract groups used a map with order as the key. When there was a collision on order, only the last value would be preserved. Changing the registration order changed which value survived the order collision.

Code snippet showing how values where put into a map and a collision on order would lose values.

    const panelGroupByOrder = new Map<string, string>();
    panelGroupData
      .filter((item): item is { order: string; groupTitle: string } =>
        Boolean(item.order && item.groupTitle)
      )
      .forEach((item) => panelGroupByOrder.set(item.order, item.groupTitle));

    return [...panelGroupByOrder.values()];

Image showing how 2 groups have the same order, thus altering test results after action registration race condition resolved. Notice values for data-group-sort-order.

Solution

PR resolves tests by replacing getPanelGroupOrder with getAddPanelFlyoutGroups and fixing order collision bug.

[nreese]

/ci

[nreese]

/ci

[nreese]

/ci

[nreese]

/ci

[nreese]

/ci

[nreese]

/ci

[nreese]

/ci

[infra-vault-gh-plugin-prod]

Pinging @elastic/kibana-presentation (Team:Presentation)

[ThomThomson]

Presentation team test only changes LGTM!

[kibanamachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 7369028

Failed CI Steps

• FTR Configs #50
• FTR Configs #190
• Scout Lane #8 - stateful-classic / default
• Scout Lane #9 - stateful-classic / default
• Scout Lane #10 - stateful-classic / default
• Scout Lane #35 - serverless-observability_complete / default

Test Failures

• [job] [logs] FTR Configs #50 / Actions APIs - Trial License/Complete Tier @serverless @serverlessQA @ess add_actions adding actions should create a case if a rule with the cases system action finds matching alerts
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Dashboards Tab
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Logs Tab
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Metadata Tab
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Metrics Tab
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Open as page and return
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Overview Tab - KPI charts and collapsible sections
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Overview Tab - Services section
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Hosts Page - Flyout - Processes Tab
• [job] [logs] Scout Lane #35 - serverless-observability_complete / default / local-serverless-observability_complete - Serverless Observability Navigation - Complete tier body - clicking body nav items sets the active link, updates breadcrumbs, and navigates
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - APM integration not installed but setup completed - Admin user
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - Collector integration is not installed - collector integration missing
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - Collector integration is not installed - Symbolizer integration is not installed
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - GET /api/console/autocomplete_entities — legacy templates - returns legacy templates when templates setting is enabled
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - Maps - maps telemetry - should return the correct telemetry values for map saved objects
• [job] [logs] Scout Lane #9 - stateful-classic / default / local-stateful-classic - Observability Landing Page (discover.isEsqlDefault enabled) - redirects to onboarding when no logs data exists
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - Profiling is setup and data is loaded - Admin user
• [job] [logs] Scout Lane #10 - stateful-classic / default / local-stateful-classic - Profiling is setup and data is loaded - Viewer user

Metrics [docs]

✅ unchanged

History

• 💛 Build #438950 was flaky f9efd31
• 💔 Build #438922 failed da86c71
• 💔 Build #438888 failed 0216f85
• 💔 Build #438872 failed a118256
• 💔 Build #438852 failed e6ecd84
• 💔 Build #438832 failed 313b17d

[delanni]

The flakiness is still there :(

[gsoldevila]

The flakiness is still there :(

Yes, I can confirm.
On my PR I get the following error, so I wonder if a simple sort of the array elements could address this:

Error: expect(received).toStrictEqual(expected) // deep equality

- Expected  - 1
+ Received  + 1

  Array [
    "visualizationsGroup",
    "controlsGroup",
    "annotation-and-navigationGroup",
-   "mlGroup",
    "logs-aiopsGroup",
+   "mlGroup",
    "legacyGroup",
  ]
    at src/platform/plugins/shared/dashboard/test/scout/ui/parallel_tests/dashboard_add_panel_flyout.spec.ts:90:24
    at src/platform/plugins/shared/dashboard/test/scout/ui/parallel_tests/dashboard_add_panel_flyout.spec.ts:88:7

cc @nreese

[gsoldevila]

I've created #268197
It's not ideal, but it should address flakiness till we find a better strategy for the 'order'.

cc @nreese @delanni

UPDATE: my PR above has different failures on the same suite:

Error: expect(received).toBe(expected) // Object.is equality

Expected: 20
Received: 17
    at src/platform/plugins/shared/dashboard/test/scout/ui/parallel_tests/dashboard_add_panel_flyout.spec.ts:96:80
    at src/platform/plugins/shared/dashboard/test/scout/ui/parallel_tests/dashboard_add_panel_flyout.spec.ts:95:7

so there's more than 1 source of flakiness

UPDATE2: I've updated #268197 to cover the 2nd case.

[nreese]

The flakiness is still there :(

Another instance of async action registration, these time in aiops. I have opened #268218 to resolve

While these flaky tests are annoying, they are exposing real race conditions in the product. I am running flaky test runner on the next fix to help limit impact on other teams.