Streams landing page improvements #215629

thomheymann · 2025-03-24T08:58:59Z

Summary

Streams landing page improvements:

Added table view for wired and classic streams
Added documents count and histogram column (loaded asynchronous)
Added time range picker to change view
Added effective retention policy column
Added pagination
Added granular loading states
Added empty state with link to onboarding page

Screenshots

…eams-landing-page-improvements

…apping'

…b.com/thomheymann/kibana into 180-streams-landing-page-improvements

flash1293

Looks pretty good already!

Some nits:

It seems like a single bar isn't rendered in the histogram, not sure why - it seems like we have the same issue on the /app/streams/<stream name>/overview page as well. We can also solve it later, but we should definitely fix - the Lens chart in Discover doesn't have the same problem.

Last 24 hours:

Last 2 hours:

The loading state looks pretty blocky - not sure how to improve, but it feels like there are some low hanging fruits here (maybe something design can help with as well?):

I noticed that when switching around or using the filter, it's very easy to do the same ESQL query over and over again. We could think about a client side cache here, but it's probably overkill since these requests are fast and also cached on the Elasticsearch side. Something for a follow-up if even that.

flash1293 · 2025-03-24T17:04:50Z

.../platform/plugins/shared/streams_app/public/components/stream_list_view/documents_column.tsx

+  indexPattern,
+  numDataPoints,
+}: {
+  timefilter: TimefilterHook; // Workaround to keep state in sync


why this workaround? I guess alternatively we could call the hook in this compoent itself, right?

We can't use the useTimerange hook more than once on the page since it uses local state and we need the time range to be in sync across the entire page.

I would suggest to create a Provider/hook that keeps the state in global context.

They should all synchronize through the global timefilter service which is part of the data plugin (this is also what retains the current time range when moving from page to page within the streams UI):

kibana/src/platform/plugins/shared/data/public/query/timefilter/use_timefilter.ts

Line 53 in 86a7103

timefilter.setTime(val);

I'm not sure about the overhead though, a provider would work too

They are only synchronised at the time filter service level.
The absolute time is set by the hook using local state so that is not in sync between different instances of the hook.

I see, the absolute time range would have small differences across the different visualizations, that is indeed weird. Passing it in works fine I guess - ideally the absolute time range should be controlled centrally in the service as well, but oh well.

flash1293 · 2025-03-24T17:06:04Z

.../platform/plugins/shared/streams_app/public/components/stream_list_view/documents_column.tsx

+  );
+
+  const docCount = allTimeseries.reduce(
+    (acc, series) => acc + series.data.reduce((acc2, item) => acc2 + (item.doc_count || 0), 0),


It doesn't matter for the classic streams, but there is a detail here we need to think about for wired streams - should we show the count for the stream and its children or just for the current layer? @LucaWintergerst any opinions here?

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/index.tsx

x-pack/platform/plugins/shared/streams_app/public/components/streams_list/index.tsx

flash1293 · 2025-03-24T17:09:54Z

x-pack/platform/plugins/shared/streams_app/public/components/stream_list_view/tree_table.tsx

+            onQuerySubmit={({ dateRange }, isUpdate) => {
+              if (dateRange && isUpdate) {
+                timefilter.setTimeRange(dateRange);
+              }


Should refreshAbsoluteTimeRange be in an else branch here?

I don't think so. I want the absolute time range to be refreshed irregardless of whether the relative time range changed. For example when a user has now-15s selected, opens the time range picker and selects the same range again I think the histograms should refresh based on that selection.

setTimeRange is implicitly updating the absolute time range as well, right? It updates the timefilter service and that in turn updates the absolute time range here:

kibana/src/platform/plugins/shared/data/public/query/timefilter/use_timefilter.ts

Line 37 in 86a7103

setAbsoluteTimeRange(() => timefilter.getAbsoluteTime());

Not sure whether it matters, just stood out to me.

The hook is currently a half way house between both approaches. Consumers need to explicitly refresh the absolute time when some interaction happens but the relative time range did not change (e.g. when selecting the same time range again or when clicking the refresh button or enabling auto-refresh). Consumers do not need to do this however when selecting a different time range. In that case the absolute time is refreshed automatically. This is fairly confusing. The behaviour should be consistent. I think the easiest approach from a consumer perspective would be to handle each of these cases for the user implicitly.

I went ahead and created a hook with context provider that solves most of the issues with the current implementation. Might make sense for the streams app search bar component to integrate with that natively so that we don't need to manually set the callback methods everywhere but we can do that incrementally when all the different use cases are clearer.

flash1293 · 2025-03-25T13:55:55Z

@thomheymann You mentioned in Slack that the loading thing is delayed by 300ms - that's right, but the way it's used is that the old number doesn't stay around:

https://github.com/elastic/kibana/pull/215629/files#diff-046d92808c06a0cdea02f0d8eaf42e27f20f6297b8985e137fb897a4f30da98bR101

Try to just hit refresh, and the old value will disappear immediately, then after 300ms the loading skeleton will appear. That's the jumpy behavior I was referring to. We should keep the old value in view until the skeleton is shown

…eams-landing-page-improvements

flash1293 · 2025-03-26T13:49:13Z

@thomheymann did you address this point somewhere? #215629 (comment)

thomheymann · 2025-03-26T18:12:10Z

@thomheymann did you address this point somewhere? #215629 (comment)

How would you expect this work instead?

flash1293 · 2025-03-27T08:43:33Z

How would you expect this work instead?

If the state switches to loading, the old value is still shown for 300ms. If the request returns within these 300ms, we update it right away, otherwise we go to the spinner. Right now if you change the time range, we immediately hide the old value and show an empty cell, then after 300ms we show the skeleton.

If the value returns quickly that causes all numbers all charts to blink.

I put up a version of what I mean here: thomheymann#2

You can go even further than that and make it nicer for edge cases, but there are diminishing returns IMHO - I was mostly triggered by the blinking.

Current state

Fast refresh time (everything blinks):

Slow refresh time (show nothing for 300ms, then the skeleton):

Keep old value for 300ms

Fast refresh time (update everything in place):

Slow refresh time (show old value for 300ms, then switch to skeleton, then to value):

flash1293

Thanks for the fix with the chart, LGTM I think it would be good to address the loading thing in my last comment, but nothing beyond that blocking the PR

elasticmachine · 2025-03-27T08:47:09Z

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

…eams-landing-page-improvements

…-fix'

elasticmachine · 2025-03-27T12:46:01Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 5d2387d

Failed CI Steps

FTR Configs #113

Test Failures

[job] [logs] FTR Configs #113 / Cloud Security Posture GET /internal/cloud_security_posture/stats CSPM Compliance Dashboard Stats API should return CSPM benchmarks V2

The CI Stats report is too large to be displayed here, check out the CI build annotation for this information.

History

💚 Build #287465 succeeded 58221a0
💛 Build #287237 was flaky 11d1db8
💛 Build #286458 was flaky d1a5bdd

kibanamachine · 2025-03-27T12:55:05Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/14106909123

## Summary Streams landing page improvements: - Added table view for wired and classic streams - Added documents count and histogram column (loaded asynchronous) - Added time range picker to change view - Added effective retention policy column - Added pagination - Added granular loading states - Added empty state with link to onboarding page ## Screenshots ![Streams-Observability-Elastic-03-24-2025_10_07_AM](https://github.com/user-attachments/assets/49379b26-3749-4707-bd1f-973650706d38) ![Streams-Observability-Elastic-03-24-2025_10_08_AM](https://github.com/user-attachments/assets/3d66c0c7-c17c-4382-a4e9-04fe5a2d1443) --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 8a29087)

kibanamachine · 2025-03-27T13:00:57Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.x`: - [Streams landing page improvements (#215629)](#215629)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Thom Heymann <[email protected]>

## Summary Streams landing page improvements: - Added table view for wired and classic streams - Added documents count and histogram column (loaded asynchronous) - Added time range picker to change view - Added effective retention policy column - Added pagination - Added granular loading states - Added empty state with link to onboarding page ## Screenshots ![Streams-Observability-Elastic-03-24-2025_10_07_AM](https://github.com/user-attachments/assets/49379b26-3749-4707-bd1f-973650706d38) ![Streams-Observability-Elastic-03-24-2025_10_08_AM](https://github.com/user-attachments/assets/3d66c0c7-c17c-4382-a4e9-04fe5a2d1443) --------- Co-authored-by: kibanamachine <[email protected]>

thomheymann added 9 commits March 17, 2025 18:13

wip

13a159f

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

2b3f1e8

…eams-landing-page-improvements

wip

952320a

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

3ad551c

…eams-landing-page-improvements

wip

6d4387b

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

d1c8fd4

…eams-landing-page-improvements

wip

86a7103

.

8859e61

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

e9d5155

…eams-landing-page-improvements

thomheymann changed the title ~~180 streams landing page improvements~~ Streams landing page improvements Mar 24, 2025

kibanamachine and others added 3 commits March 24, 2025 09:11

[CI] Auto-commit changed files from 'node scripts/styled_components_m…

d84d13e

…apping'

remove unused translations

ee504ae

Merge branch '180-streams-landing-page-improvements' of https://githu…

d1a5bdd

…b.com/thomheymann/kibana into 180-streams-landing-page-improvements

elastic deleted a comment from elasticmachine Mar 24, 2025

thomheymann marked this pull request as ready for review March 24, 2025 12:45

thomheymann requested a review from a team as a code owner March 24, 2025 12:45

thomheymann added backport:version Backport to applied version labels v8.19.0 v9.1.0 release_note:skip Skip the PR/issue when compiling release notes labels Mar 24, 2025

flash1293 reviewed Mar 24, 2025

View reviewed changes

thomheymann added 2 commits March 26, 2025 10:04

Added suggestions from code and design review

46df32f

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

11d1db8

…eams-landing-page-improvements

thomheymann requested a review from flash1293 March 26, 2025 10:42

Timefilter provider

10f1774

Fix broken charts

58221a0

flash1293 approved these changes Mar 27, 2025

View reviewed changes

flash1293 added Team:obs-onboarding Observability Onboarding Team Feature:Streams This is the label for the Streams Project labels Mar 27, 2025

flash1293 mentioned this pull request Mar 27, 2025

[Streams 🌊] add toggle for short stream names #211813

Closed

9 tasks

thomheymann and others added 4 commits March 27, 2025 10:29

Remove delay for rendering loading states

5d030f9

Merge branch 'main' of https://github.com/elastic/kibana into 180-str…

e52f9b9

…eams-landing-page-improvements

Increase default pagination

128bc5c

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

5d2387d

…-fix'

thomheymann enabled auto-merge (squash) March 27, 2025 11:05

thomheymann merged commit 8a29087 into elastic:main Mar 27, 2025
9 checks passed

kibanamachine mentioned this pull request Mar 27, 2025

[8.x] Streams landing page improvements (#215629) #216166

Merged

Streams landing page improvements #215629

Streams landing page improvements #215629

Uh oh!

Conversation

thomheymann commented Mar 24, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Screenshots

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thomheymann Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flash1293 commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flash1293 commented Mar 26, 2025

Uh oh!

thomheymann commented Mar 26, 2025

Uh oh!

flash1293 commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current state

Keep old value for 300ms

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 27, 2025

Uh oh!

Uh oh!

elasticmachine commented Mar 27, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

History

Uh oh!

kibanamachine commented Mar 27, 2025

Uh oh!

kibanamachine commented Mar 27, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

thomheymann commented Mar 24, 2025 •

edited by kibanamachine

Loading

thomheymann Mar 25, 2025 •

edited

Loading

flash1293 commented Mar 25, 2025 •

edited

Loading

flash1293 commented Mar 27, 2025 •

edited

Loading