[Streams 🌊] Listing and overview page improvements by rStelmach · Pull Request #223603 · elastic/kibana

rStelmach · 2025-06-12T16:25:36Z

📝 Summary

Closes: #238

This pull request introduces enhancements to the classic streams, focusing on visual updates, improved user interface elements, and the addition of new assets. Things that were done in this PR :

Improve page template to have transparent background and short description
Replaced old images with new ones
Introduced new empty state for streams list
Added sorting to streams list table
Improved loading indicators
Aligned streams tree table with designs and added loading indicators
Added tooltips to modify streams tabs as well as retention badge and stream-type badge
Modified Component templates table according to new designs

🎨 Previews

Empty State Before:
Empty State After:

Streams List with no data Before:

Streams List with no data After:

Streams List Before:

Streams List After:

Component Templates Before:

Component Templates After:

Small demo of streams being loaded

loading.demo.mov

🔧 How to test

In dev console in kibana run POST kbn:/api/streams/_enable. This will enable the streams UI and put mappings and pipelines for the main logs stream into place
Reload the page
Navigate to the streams app in navigation
Ingesting data through synthrace: node scripts/synthtrace.js sample_logs --live --kibana=http://elastic:changeme@localhost:5601 --target=http://elastic:changeme@localhost:9200 --liveBucketSize=1000

For more info check this README file

Also please check the issue and discussion for more context

❓ Discussion

if you think that there is something else that needs improvements for classic streams, let me know so we can include it in this PR

🖌 Designs

Figma
New Images

…anage streams tabs

…ace old images with new ones

elasticmachine · 2025-06-13T14:36:32Z

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

thomheymann

Haven't gone through the code yet but couple questions regarding the UI below

Streams listing page

Streams details page

thomheymann · 2025-06-17T12:27:39Z

There's an issue with the retention policy badges. Some open in new windows and some dont. I think this should be consistent:

...s_app/public/components/data_management/stream_detail_management/ingest_pipeline_details.tsx

...ns/shared/streams_app/public/components/data_management/stream_detail_management/wrapper.tsx

thomheymann · 2025-06-17T12:48:14Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_detail_view/index.tsx

+                <EuiToolTip
+                  position="top"
+                  title={i18n.translate('xpack.streams.badges.classic.title', {
+                    defaultMessage: 'Classic Stream',
+                  })}
+                  content={i18n.translate('xpack.streams.badges.classic.description', {
+                    defaultMessage:
+                      'Classic streams are based on existing data streams and may not support all Streams features like custom re-routing',
+                  })}
+                >
+                  <ClassicStreamBadge />
+                </EuiToolTip>


Could the tooltip be part of the badge so we get it for free wherever we display it?

@rStelmach What do you think?

My bad, I removed it only from index. Right now it's removed also from wrapper.

I left the wrapped tooltip around <LifecycleBadge>, I think it's cleaner than placing the wrapper in every if case inside the component, but let me know if you think that it should be done the same way.

.../platform/plugins/shared/streams_app/public/components/stream_list_view/documents_column.tsx

thomheymann · 2025-06-17T12:50:19Z

.../platform/plugins/shared/streams_app/public/components/stream_list_view/documents_column.tsx

            `}
          >
-            <EuiI18nNumber value={docCount} />
+            {hasData ? <EuiI18nNumber value={docCount} /> : 'N/A'}


N/A should be translated

...m/plugins/shared/streams_app/public/components/stream_list_view/streams_list_empty_state.tsx

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/tree_table.tsx

thomheymann · 2025-06-17T13:41:04Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/tree_table.tsx

+  const sortedRoots = React.useMemo(() => {
+    return [...enrichedTree].sort((a, b) => {
+      const av = a[sortField];
+      const bv = b[sortField];
+      if (typeof av === 'string' && typeof bv === 'string') {
+        return sortDirection === 'asc' ? av.localeCompare(bv) : bv.localeCompare(av);
+      }
+      if (typeof av === 'number' && typeof bv === 'number') {
+        return sortDirection === 'asc' ? av - bv : bv - av;
+      }
+      return 0;
+    });
+  }, [enrichedTree, sortField, sortDirection]);


The sorting logic does not work correctly. I think in this case "logs.linux" should come before "logs.apache":

The sorting is still broken. The name column has the same issue we discussed for the documents column in that it only sorts root level streams.

Right now it should work

With the current logic we start off with a flat list of streams (coming from GET /internal/streams). We are then turning that into a deeply nested object, walking over the entire tree structure to sort nodes and then walking over the entire structure again to flatten it back to a flat list and enrich it with meta data.

Is all that necessary or can we optimise the code to reduce complexity?

I think we should be able to derive any required fields we need in a single iteration and then let EuiMemoryTable handle the sorting:

const enrichedStreams = streamsResponse.map(enrichWithMetaDataAndSortKeys)

What do you think?

I tried to do it in a single iteration and pass it to EuiMemoryTable but that component is unaware of parent/child boundaries, and it mixes children from different roots.

I tried to simplified the logic. Now items is built in a single depth-first walk that flattens the tree and adds level + root-level meta in one go.

What do you think of this change ? Is it a bit cleaner?

Yeh but I think the sort order is inverted. Ascending means 0-9/A-Z but here it seems to be the other way around:

Can you also add unit tests for the sorting logic to ensure it works correctly?

gbamparop · 2025-06-18T11:55:07Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/index.tsx

+          })}{' '}
+          <EuiLink
+            target="_blank"
+            href="https://www.elastic.co/docs/solutions/observability/logs/streams/streams"


Can we use the doc links service instead?

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/index.tsx

…dges to wrap lifecycle badge

… sorting

thomheymann

Thanks, the code looks good and much clearer now.

For the unit tests, I don't think we need to be clever here or try and make this dry.

The function is supposed to sort an entire list of streams so our assertions should also verify the entire list. If you start filtering you create blind spots where the sort order will not be tested correctly.

We need the following 4 test scenarios:

Sort by name (asc)
Sort by name (desc)
Sort by retention time (asc)
Sort by retention time (desc)

We need to check that the full list has been sorted correctly, not just parts.

thomheymann · 2025-06-20T10:17:53Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/utils.test.ts

+    const rows = buildStreamRows(allStreams, 'nameSortKey', 'asc' as Direction);
+    expect(rows.map((r) => r.name)).toEqual([
+      'logs-a',
+      'logs-a.child1',
+      'logs-a.child2',
+      'logs-a.child3',
+      'logs-b',
+      'logs-b.child1',
+      'logs-b.child2',
+      'logs-b.child3',
+      'metrics-c',
+      'metrics-c.child1',
+      'metrics-c.child2',
+      'metrics-c.child3',
+    ]);


Can you add a case for desc order?

thomheymann · 2025-06-20T10:21:03Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/utils.test.ts

+    expect(rows.map((r) => r.name).slice(0, 5)).toEqual([
+      'logs-b',
+      'logs-b.child3',
+      'logs-b.child2',
+      'logs-b.child1',
+      'metrics-c',
+    ]);


Why are you slicing the result?

It's quite hard to see what the expected behaviour here is without seeing the retention time. Can you include that? (e.g. rows.map((r) => ${r.name} (${retentionTime})))

thomheymann · 2025-06-20T10:24:56Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/utils.test.ts

+  it('children are sorted by retention asc inside each root', () => {
+    const rows = buildStreamRows(allStreams, 'retentionMs', 'asc' as Direction);
+    const aChildren = rows
+      .filter((r) => r.name.startsWith('logs-a.') && r.level === 1)
+      .map((r) => r.name);
+    expect(aChildren).toEqual(['logs-a.child2', 'logs-a.child3', 'logs-a.child1']);
+  });


What is this assertion checking?

…apping'

thomheymann

LGTM! 🥳

elasticmachine · 2025-06-20T13:21:11Z

💚 Build Succeeded

Buildkite Build
Commit: f9d4e1c

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`streamsApp`	460	471	+11

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiAssistantManagementSelection`	77.1KB	77.2KB	+56.0B
`lists`	123.8KB	123.9KB	+56.0B
`streamsApp`	541.0KB	553.1KB	+12.1KB
total			+12.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	457.5KB	457.6KB	+56.0B

Unknown metric groups

async chunk count

id	before	after	diff
`streamsApp`	8	18	+10

miscellaneous assets size

id	before	after	diff
`streamsApp`	60.6KB	304.0KB	⚠️ +243.3KB

History

💚 Build #310035 succeeded e6d994f
💚 Build #309557 succeeded 7c885b6
💛 Build #309433 was flaky ae435f2
💚 Build #309314 succeeded 68700ab
💚 Build #308891 succeeded e703a22
💛 Build #308383 was flaky ffe3b96

kibanamachine · 2025-06-20T13:24:33Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15779974687

kibanamachine · 2025-06-20T13:30:52Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 223603

Questions ?

Please refer to the Backport tool documentation

rStelmach · 2025-06-20T14:47:20Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

## 📝 Summary Closes: [elastic#238](elastic/streams-program#238) This pull request introduces enhancements to the classic streams, focusing on visual updates, improved user interface elements, and the addition of new assets. Things that were done in this PR : - Improve page template to have transparent background and short description - Replaced old images with new ones - Introduced new empty state for streams list - Added sorting to streams list table - Improved loading indicators - Aligned streams tree table with designs and added loading indicators - Added tooltips to modify streams tabs as well as retention badge and stream-type badge - Modified `Component templates` table according to new designs ## 🎨 Previews Empty State Before: ![empty state before](https://github.com/user-attachments/assets/120b36b7-ff7b-4cb4-ab70-40908aa62f95) Empty State After: ![empty_state after](https://github.com/user-attachments/assets/f3f7529a-f2f8-4900-86b0-1bc938ed5264) Streams List with no data Before: ![streams list no data before](https://github.com/user-attachments/assets/c98d6095-88b0-4eaf-8c04-d1934cd1406a) Streams List with no data After: ![streams list no data](https://github.com/user-attachments/assets/00612e5d-5abb-48b0-9837-ca769bd96b66) Streams List Before: ![streams list with data before](https://github.com/user-attachments/assets/2f66bc32-3cf8-43ce-989c-91d5b72d4622) Streams List After: ![streams_list with data](https://github.com/user-attachments/assets/4f6a5987-de75-49b2-8e0e-86dfef130557) Component Templates Before: ![Component templates before](https://github.com/user-attachments/assets/5091eaa2-7983-48f0-ba68-808692235be4) Component Templates After: ![component templates after](https://github.com/user-attachments/assets/cc590c88-6170-4bc1-a18d-faf870163098) ### Small demo of streams being loaded https://github.com/user-attachments/assets/f4a9ec48-ee01-4791-b874-03afeb7f719a ## 🔧 How to test - In dev console in kibana run POST kbn:/api/streams/_enable. This will enable the streams UI and put mappings and pipelines for the main logs stream into place - Reload the page - Navigate to the streams app in navigation - Ingesting data through synthrace: `node scripts/synthtrace.js sample_logs --live --kibana=http://elastic:changeme@localhost:5601 --target=http://elastic:changeme@localhost:9200 --liveBucketSize=1000` For more info check this [README](https://github.com/elastic/streams-program) file Also please check the [issue and discussion](elastic/streams-program#238) for more context ## ❓ Discussion if you think that there is something else that needs improvements for classic streams, let me know so we can include it in this PR ## 🖌 Designs [Figma](https://www.figma.com/design/MgwQzRrh2CWFSE711GEW1o/Streams-Index-and-Onboarding?node-id=1343-139771&m=dev&focus-id=1344-51082) [New Images ](https://elasticsearch.app.box.com/s/37d3su7j500n9ax6666pv1hbckycjjr0/folder/319936922134) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 4741e67) # Conflicts: # src/platform/packages/shared/kbn-doc-links/src/get_doc_links.ts # x-pack/platform/plugins/shared/streams_app/public/components/data_management/stream_detail_management/wired.tsx # x-pack/platform/plugins/shared/streams_app/public/components/data_management/stream_detail_management/wrapper.tsx # x-pack/platform/plugins/shared/streams_app/public/components/stream_badges/index.tsx # x-pack/platform/plugins/shared/streams_app/tsconfig.json

…224715) # Backport This will backport the following commits from `main` to `8.19`: - [[Streams 🌊] Listing and overview page improvements (#223603)](#223603)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## 📝 Summary Closes: [elastic#238](elastic/streams-program#238) This pull request introduces enhancements to the classic streams, focusing on visual updates, improved user interface elements, and the addition of new assets. Things that were done in this PR : - Improve page template to have transparent background and short description - Replaced old images with new ones - Introduced new empty state for streams list - Added sorting to streams list table - Improved loading indicators - Aligned streams tree table with designs and added loading indicators - Added tooltips to modify streams tabs as well as retention badge and stream-type badge - Modified `Component templates` table according to new designs ## 🎨 Previews Empty State Before: ![empty state before](https://github.com/user-attachments/assets/120b36b7-ff7b-4cb4-ab70-40908aa62f95) Empty State After: ![empty_state after](https://github.com/user-attachments/assets/f3f7529a-f2f8-4900-86b0-1bc938ed5264) Streams List with no data Before: ![streams list no data before](https://github.com/user-attachments/assets/c98d6095-88b0-4eaf-8c04-d1934cd1406a) Streams List with no data After: ![streams list no data](https://github.com/user-attachments/assets/00612e5d-5abb-48b0-9837-ca769bd96b66) Streams List Before: ![streams list with data before](https://github.com/user-attachments/assets/2f66bc32-3cf8-43ce-989c-91d5b72d4622) Streams List After: ![streams_list with data](https://github.com/user-attachments/assets/4f6a5987-de75-49b2-8e0e-86dfef130557) Component Templates Before: ![Component templates before](https://github.com/user-attachments/assets/5091eaa2-7983-48f0-ba68-808692235be4) Component Templates After: ![component templates after](https://github.com/user-attachments/assets/cc590c88-6170-4bc1-a18d-faf870163098) ### Small demo of streams being loaded https://github.com/user-attachments/assets/f4a9ec48-ee01-4791-b874-03afeb7f719a ## 🔧 How to test - In dev console in kibana run POST kbn:/api/streams/_enable. This will enable the streams UI and put mappings and pipelines for the main logs stream into place - Reload the page - Navigate to the streams app in navigation - Ingesting data through synthrace: `node scripts/synthtrace.js sample_logs --live --kibana=http://elastic:changeme@localhost:5601 --target=http://elastic:changeme@localhost:9200 --liveBucketSize=1000` For more info check this [README](https://github.com/elastic/streams-program) file Also please check the [issue and discussion](elastic/streams-program#238) for more context ## ❓ Discussion if you think that there is something else that needs improvements for classic streams, let me know so we can include it in this PR ## 🖌 Designs [Figma](https://www.figma.com/design/MgwQzRrh2CWFSE711GEW1o/Streams-Index-and-Onboarding?node-id=1343-139771&m=dev&focus-id=1344-51082) [New Images ](https://elasticsearch.app.box.com/s/37d3su7j500n9ax6666pv1hbckycjjr0/folder/319936922134) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

initial version

d2132ee

rStelmach changed the title ~~initial version~~ Streams Listing and overview page improvements Jun 12, 2025

rStelmach added 3 commits June 13, 2025 14:35

improve overview table styles and advanced section.Added tooltip to m…

c711e7b

…anage streams tabs

fix logic for keeping children together

0c65be1

replace old empty state component with new one for streams list. Repl…

4815043

…ace old images with new ones

rStelmach changed the title ~~Streams Listing and overview page improvements~~ [Streams 🌊] Listing and overview page improvements Jun 13, 2025

rStelmach marked this pull request as ready for review June 13, 2025 14:35

rStelmach requested review from a team as code owners June 13, 2025 14:35

rStelmach added Team:obs-onboarding Observability Onboarding Team backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Jun 13, 2025

rStelmach added Feature:Streams This is the label for the Streams Project release_note:enhancement labels Jun 13, 2025

Merge branch 'main' into streams-listing-and-overview-page-improvements

ffe3b96

thomheymann reviewed Jun 13, 2025

View reviewed changes

rStelmach added 2 commits June 16, 2025 16:25

optimize calls. introduce folding and unfloding streams trees

e703a22

remove collapsing logic. adjust margin in components table

68700ab

thomheymann reviewed Jun 17, 2025

View reviewed changes

rStelmach added 3 commits June 17, 2025 15:58

Merge branch 'main' into streams-listing-and-overview-page-improvements

ae435f2

remove logic for sorting documents. adjust styles, rename empty states

7c885b6

enable sorting within the streams trees

05a874b

gbamparop reviewed Jun 18, 2025

View reviewed changes

thomheymann reviewed Jun 18, 2025

View reviewed changes

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/index.tsx Show resolved Hide resolved

rStelmach added 3 commits June 18, 2025 15:07

remove tooltip wrapped around ClassicStreamsBadge

a984b33

simplyfy and break down sorting logic

5b089ab

add streams documentation link to docLinks service

e6d994f

rStelmach requested a review from a team as a code owner June 18, 2025 14:53

mdbirnstiehl approved these changes Jun 18, 2025

View reviewed changes

rStelmach added 2 commits June 20, 2025 11:04

merge main, resolve conflicts, create a tooltip wrapper in stream ba…

dae231a

…dges to wrap lifecycle badge

extract sorting logic from tree_table to utils. create unit tests for…

219b886

… sorting

thomheymann reviewed Jun 20, 2025

View reviewed changes

rStelmach and others added 3 commits June 20, 2025 12:54

simplify tests aand cover 4 major cases

c073236

add retention time for a better test readibiliy

fd5cff6

[CI] Auto-commit changed files from 'node scripts/styled_components_m…

f9d4e1c

…apping'

thomheymann approved these changes Jun 20, 2025

View reviewed changes

rStelmach merged commit 4741e67 into elastic:main Jun 20, 2025
10 checks passed

kibanamachine mentioned this pull request Jun 20, 2025

[Response Ops][Connectors] New xpack.actions.email.services.enabled Kibana setting #223363

Merged

rStelmach mentioned this pull request Jun 20, 2025

[8.19] [Streams 🌊] Listing and overview page improvements (#223603) #224715

Merged

This was referenced Jun 20, 2025

[Response Ops][Reporting] Scheduled report API license check #224737

Merged

[Streams 🌊] Improve warning on missing simulation samples #224405

Merged

Conversation

rStelmach commented Jun 12, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Summary

🎨 Previews

Small demo of streams being loaded

🔧 How to test

❓ Discussion

🖌 Designs

Uh oh!

elasticmachine commented Jun 13, 2025

Uh oh!

thomheymann left a comment

Choose a reason for hiding this comment

Streams listing page

Streams details page

Uh oh!

thomheymann commented Jun 17, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

thomheymann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomheymann left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 20, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

miscellaneous assets size

History

Uh oh!

Uh oh!

kibanamachine commented Jun 20, 2025

Uh oh!

kibanamachine commented Jun 20, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

rStelmach commented Jun 20, 2025

💚 All backports created successfully

Questions ?

rStelmach commented Jun 12, 2025 •

edited by kibanamachine

Loading