[Streams] upgrade tests by CoenWarmer · Pull Request #249617 · elastic/kibana

CoenWarmer · 2026-01-19T16:44:02Z

Summary

This updates Streams App Scout tests.

Details

Adds Scout API tests for relevant areas
Removes Scout UI tests for areas now more reliably covered by API tests
Attempts to address low hanging fruit with regards to flaky UI tests for Streams App

UI Tests:

Category	Before	After	Reduction
Data Retention	13 tests (8 files)	8 tests (7 files)	-5 tests, -1 file
Data Routing	~24 tests	~24 tests	Added comments only
Data Processing	~11 tests	~9 tests	-2 tests
Total	~48 tests	~41 tests	~15% reduction

API Tests:

lifecycle_retention.spec.ts - 16 tests covering DSL retention, inheritance, switching
routing_fork_stream.spec.ts - 24 tests covering all condition types (eq, neq, and, or, not, etc.)
processing_simulate.spec.ts - 28 tests covering all processor types (grok, dissect, date, rename, etc.)
schema_field_mapping.spec.ts - 27 tests covering field type simulation

Kerry350

Nice work 👏

Just a question about the location: for the API tests should they still live in the streams_app plugin, or should those move to the streams plugin since that's where the server side code lives?

CoenWarmer · 2026-01-22T12:15:49Z

Nice work 👏
ty :)

Just a question about the location: for the API tests should they still live in the streams_app plugin, or should those move to the streams plugin since that's where the server side code lives?

Good question. I think it probably makes more sense to move them to streams plugin as the tests will be closest to their implementations.

…-tests

dmlemeshko · 2026-01-23T13:21:05Z

+      // Wait for the data view switcher to be available before selecting
+      await page.locator('[data-test-subj*="dataView-switch-link"]').waitFor({
+        state: 'visible',
+        timeout: 30_000,
+      });


Waiting for this element is already handled inside pageObjects.discover.selectDataView:

const currentValue = await this.page.testSubj.innerText('*dataView-switch-link');

will wait for 10 sec (default) to locate the element.

What I used in other tests, was these 2 methods:

await pageObjects.discover.waitUntilSearchingHasFinished(); await pageObjects.discover.waitForHistogramRendered();

you can use both or at least wait for histogram to render before selecting data view.

Thanks @dmlemeshko, that's very helpful

dmlemeshko · 2026-01-23T13:21:44Z

+      // Wait for the data grid to be fully rendered
+      await page.locator('[data-test-subj="discoverDocTable"]').waitFor({
+        state: 'visible',
+        timeout: 30_000,
+      });


already done with await pageObjects.discover.waitForDocTableRendered()

dmlemeshko · 2026-01-23T13:22:26Z

+      // Wait for the data grid to be fully rendered
+      await page.locator('[data-test-subj="discoverDocTable"]').waitFor({
+        state: 'visible',
+        timeout: 30_000,
+      });


await pageObjects.discover.waitForDocTableRendered()

dmlemeshko · 2026-01-23T13:23:56Z

-      // Verify the doc viewer flyout is open
-      await expect(page.getByTestId('kbnDocViewer')).toBeVisible();
+      // Verify the doc viewer flyout is open (with extended timeout for flyout animation)
+      await expect(page.getByTestId('kbnDocViewer')).toBeVisible({ timeout: 30_000 });


what do you think about adding a method in https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-scout/src/playwright/page_objects/discover_app.ts

dmlemeshko · 2026-01-23T13:26:10Z

      await page.getByTestId('streamsAppContentAddFieldButton').click();

-      // Wait `/fields_metadata` so that we are sure the ECS/Otel mapping recommendations are available
-      await page.waitForResponse(
-        (response) => response.url().includes('/fields_metadata') && response.ok()
-      );
-
      // Add an Otel field that should have type recommendation (IP type)
      const ecsFieldName = 'resource.attributes.host.ip';
      await pageObjects.streams.typeFieldName(ecsFieldName);


Do we need any extra wait between clicking Add field and start typing to replace removed logic? We saw in the past event handler not being attached fast enough and Kibana acting weird.

dmlemeshko · 2026-01-23T13:30:35Z

+  try {
+    await esClient.index({
+      index: 'logs',
+      document: {
+        '@timestamp': new Date().toISOString(),
+        message: 'Test document for streams API tests',
+      },
+      refresh: true,
+    });
+    log.debug('[setup] Test document indexed successfully');
+  } catch (error) {
+    log.debug(`[setup] Failed to index test document: ${error}`);
+  }


question: is it ok to proceed with tests run even if indexing failed? The same question is related to enabling streams

Makes sense to mark that as a failure.

dmlemeshko · 2026-01-23T13:32:37Z

+  try {
+    await kbnClient.request({
+      method: 'POST',
+      path: '/api/streams/_enable',
+    });
+    log.debug('[setup] Streams enabled successfully');
+  } catch (error) {
+    log.debug(`[setup] Streams may already be enabled: ${error}`);
+  }


Afaik we don't need try/catch as it won't throw error if Streams've been already enabled.

We already have this code working:

kibana/src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/streams/index.ts

Lines 55 to 58 in 99d641f

await kbnClient.request({

method: 'POST',

path: `${basePath}/api/streams/_enable`,

});

dmlemeshko · 2026-01-23T13:38:25Z

+api/
+├── playwright.config.ts          # Scout API test configuration
+├── constants.ts                  # Common headers and constants
+├── fixtures/
+│   ├── index.ts                 # Test fixtures extending apiTest
+│   └── constants.ts             # User roles and API headers
+├── services/
+│   └── streams_api_service.ts   # Streams API helper service
+└── tests/
+    ├── global.setup.ts          # Global setup (enables Streams)
+    ├── processing_simulate.spec.ts
+    ├── routing_fork_stream.spec.ts
+    ├── schema_field_mapping.spec.ts
+    └── lifecycle_retention.spec.ts


Happy the day has come :) After UI tests count crossed 200 I was hoping someone will change the pattern and add API tests for Streams. Thank you @CoenWarmer 👏

dmlemeshko

.buildkite/scout_ci_config.yml changes LGTM, I left a few minor comments.

CoenWarmer · 2026-01-23T13:42:00Z

Thanks for the review @dmlemeshko, very helpful!

elasticmachine · 2026-01-23T18:25:54Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 8493f4f

Failed CI Steps

Scout: [ platform / fleet ] plugin

Test Failures

[job] [logs] Scout: [ platform / fleet ] plugin / Create Integration is not accessible when user is basic
[job] [logs] Scout: [ platform / fleet ] plugin / stateful - User with basic license should hit License Paywall - Create Integration is not accessible when user is basic

Metrics [docs]

✅ unchanged

History

💔 Build #385434 failed 68f3ea8
💔 Build #385366 failed b067a75
💔 Build #385293 failed 7a6331a
💔 Build #385272 failed c88e017
💔 Build #385172 failed f17339e
💔 Build #384962 failed 730081f

CoenWarmer added 3 commits January 19, 2026 17:26

Add Scout API tests for Streams

ee76445

Remove Scout UI tests now covered by Scout API tests

a6802e3

Deflakify Scout UI tests

de3fecc

CoenWarmer requested review from a team as code owners January 19, 2026 16:44

CoenWarmer added backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes labels Jan 19, 2026

CoenWarmer changed the title ~~Streams upgrade tests~~ [Streams] upgrade tests Jan 19, 2026

kibanamachine and others added 3 commits January 19, 2026 16:53

Changes from node scripts/lint_ts_projects --fix

19dfadf

Changes from node scripts/regenerate_moon_projects.js --update

20e3c39

Update fixture for Serverless

88b9530

Kerry350 self-requested a review January 21, 2026 10:05

CoenWarmer mentioned this pull request Jan 21, 2026

[scout tests] fix flaky FieldName typing for MKI #249754

Merged

Kerry350 approved these changes Jan 22, 2026

View reviewed changes

CoenWarmer and others added 7 commits January 22, 2026 13:33

Move API tests to streams

92c5b5e

Merge branch 'main' of github.com:elastic/kibana into streams-upgrade…

31a5c57

…-tests

Changes from node scripts/lint_packages --fix

2659835

Changes from node scripts/regenerate_moon_projects.js --update

6b7ce63

Fix

730081f

Merge branch 'main' into streams-upgrade-tests

f17339e

Cover tests in tsconfig

c88e017

CoenWarmer enabled auto-merge (squash) January 23, 2026 09:13

kibanamachine and others added 3 commits January 23, 2026 09:24

Changes from node scripts/lint_ts_projects --fix

5cde080

Changes from node scripts/regenerate_moon_projects.js --update

7a6331a

Add Streams to scout_ci_config

b067a75

CoenWarmer requested a review from a team as a code owner January 23, 2026 11:45

Fix tests

0bed8fe

dmlemeshko reviewed Jan 23, 2026

View reviewed changes

dmlemeshko approved these changes Jan 23, 2026

View reviewed changes

Bits and bops

68f3ea8

CoenWarmer added 2 commits January 23, 2026 17:23

Update tests

efc9f8c

Merge branch 'main' into streams-upgrade-tests

8493f4f

CoenWarmer merged commit 40ae3b8 into elastic:main Jan 23, 2026
15 of 16 checks passed

kibanamachine added the v9.4.0 label Jan 23, 2026

	await kbnClient.request({
	method: 'POST',
	path: `${basePath}/api/streams/_enable`,
	});

Conversation

CoenWarmer commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details

UI Tests:

API Tests:

Uh oh!

Kerry350 left a comment

Choose a reason for hiding this comment

Uh oh!

CoenWarmer commented Jan 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko left a comment

Choose a reason for hiding this comment

Uh oh!

CoenWarmer commented Jan 23, 2026

Uh oh!

Uh oh!

elasticmachine commented Jan 23, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

CoenWarmer commented Jan 19, 2026 •

edited

Loading

dmlemeshko Jan 23, 2026 •

edited

Loading

dmlemeshko Jan 23, 2026 •

edited

Loading

dmlemeshko Jan 23, 2026 •

edited

Loading