[Logs] Replace log stream component by achyutjhunjhunwala · Pull Request #219425 · elastic/kibana

achyutjhunjhunwala · 2025-04-28T15:39:46Z

Summary

closes https://github.com/elastic/logs-dev/issues/111
closes #200668
closes https://github.com/elastic/opentelemetry-dev/issues/792
closes #89042
closes #221746
closes #91717
closes #91717

The PR does the following

Enhances the enableDocumentViewer API in the Embeddable to also hide Quality issues and Stacktrace Columns for obs context when the expand icon is also hidden
Add a Columns Props the the embeddable so that a new profile is not required to be created when custom columns are required
Replaces Log Stream component in all remaining places
APM - Transactions Details Page

Logs tab in the hosts' view page

Logs Tab in Host Detail view

Logs Tab in the Flyout from the main Hosts page

Fleet Agent Logs

Logs Categories Page

What's Pending ?

Code for the old Log Stream Component will be cleaned up in the following PR

…getting truncated

…s the flyout which can be opened from hosts list view

…apping'

…urce-definitions/scripts/fix-location-collection.ts'

…date LogStream Component in Category page as well

… with Saved Search

juliaElastic

LGTM

tonyghiani

LGTM!

davismcphee

The log stream embeddable approach seems ok overall, but I'd really like to avoid special paths in the Discover embeddable to account for it. Could we please try something similar to my suggestion to avoid it instead?

src/platform/plugins/shared/discover/public/embeddable/initialize_search_embeddable_api.tsx

davismcphee · 2025-06-18T00:13:09Z

src/platform/plugins/shared/discover/public/embeddable/initialize_search_embeddable_api.tsx

+  if (!dataView) {
+    try {
+      // Try to load with the "All logs" data view ID. This is faster
+      dataView = await dataService.dataViews.get(ALL_LOGS_DATA_VIEW_ID);


I wouldn't block the PR over it, but I don't really think this is necessary. It causes a 404 response whenever the cached data view doesn't exist yet, and using dataViews.create will already check the cache for a data view with a matching ID before creating one by default. The only perf hit incurred by skipping this and always calling create would be from getFlattenedLogSources, which I imagine is negligible.

Thank you for clarifying this.

Also the logsDataAccess pluggin is an optional dependency in Discover. Do you think it should be a requiredPluggin ? That would make certain checks easier

That's a great question, and tbh I'm not 100% sure 🙃 All of our required plugins are outside of x-pack so maybe there's some restriction there, but some of the optional ones are actually required for Discover to work (e.g. Lens). So I think we can expect it will always be there at runtime, but your fallback logic will at least protect against it regardless. No need to wait for this PR, but now that you raised it I'm gonna ask operations.

davismcphee · 2025-06-18T00:16:59Z

src/platform/plugins/shared/discover/public/plugin.tsx

    });
+
+    // We register the saved search embeddable factory for the log stream embeddable to support old log stream panels.
+    plugins.embeddable.registerReactEmbeddableFactory('LOG_STREAM_EMBEDDABLE', async () => {


I don't love the idea of having the legacy o11y log stream embeddable registered from Discover, but I also can't think of a good alternative considering it depends on the Discover embeddable factory... We'll just be sure to ping the Logs UX team if we get any bug reports 😄

Yeah unfortunately, i had to do this. I tried porting dependencies to Infra but it turned out to be a nightmare so i had to do it the other way.

Good part is we have Telemetry captured for the number of clusters using this Panel. Once we see this going to 0, we can plan to remove it completely may be.

Agreed, definitely not worth trying to port it, and good to know we have telemetry on this.

src/platform/test/functional/apps/discover/embeddable/log_stream_dashboard_saved_object.ndjson

…le not polluting Discover Saved Search embeddable

davismcphee

Data Discovery changes LGTM 👍 Thanks for making adjustments to the embeddable approach!

davismcphee · 2025-06-18T19:25:51Z

...atform/plugins/shared/discover/public/embeddable/get_legacy_log_stream_embeddable_factory.ts

+    type: LEGACY_LOG_STREAM_EMBEDDABLE,
+    buildEmbeddable: async ({ initialState, ...restParams }) => {
+      const searchSource = await discoverServices.data.search.searchSource.create();
+      let fallbackPattern = 'logs-*-*';


Good idea with the fallback to ensure we always have an index pattern available 👍

davismcphee · 2025-06-18T20:51:40Z

src/platform/plugins/shared/discover/public/embeddable/initialize_search_embeddable_api.tsx

+  if (!dataView) {
+    try {
+      // Try to load with the "All logs" data view ID. This is faster
+      dataView = await dataService.dataViews.get(ALL_LOGS_DATA_VIEW_ID);


That's a great question, and tbh I'm not 100% sure 🙃 All of our required plugins are outside of x-pack so maybe there's some restriction there, but some of the optional ones are actually required for Discover to work (e.g. Lens). So I think we can expect it will always be there at runtime, but your fallback logic will at least protect against it regardless. No need to wait for this PR, but now that you raised it I'm gonna ask operations.

davismcphee · 2025-06-18T20:53:10Z

src/platform/plugins/shared/discover/public/plugin.tsx

    });
+
+    // We register the saved search embeddable factory for the log stream embeddable to support old log stream panels.
+    plugins.embeddable.registerReactEmbeddableFactory('LOG_STREAM_EMBEDDABLE', async () => {


Agreed, definitely not worth trying to port it, and good to know we have telemetry on this.

kibanamachine · 2025-06-19T11:27:07Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15756729781

elasticmachine · 2025-06-19T11:27:29Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`discover`	1320	1321	+1
`fleet`	1179	1249	+70
`infra`	1470	1443	-27
total			+44

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	2.6MB	2.6MB	+280.0B
`discover`	1.1MB	1.1MB	+1.5KB
`fleet`	1.8MB	1.8MB	+2.4KB
`infra`	1.2MB	1.1MB	-61.8KB
total			-57.6KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`apm`	35.5KB	35.4KB	-140.0B
`discover`	21.5KB	21.8KB	+229.0B
`fleet`	168.2KB	168.2KB	+38.0B
`infra`	50.3KB	50.0KB	-354.0B
total			-227.0B

Unknown metric groups

async chunk count

id	before	after	diff
`fleet`	9	10	+1
`infra`	30	29	-1
total			-0

History

💛 Build #310022 was flaky c076e74
💛 Build #309825 was flaky 9f1eaa1
💚 Build #309173 succeeded afd4f42
💛 Build #309086 was flaky 58094fb
💚 Build #308952 succeeded 9634678
💚 Build #308651 succeeded 195ef4b

cc @achyutjhunjhunwala

kibanamachine · 2025-06-19T11:34:03Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 219425

Questions ?

Please refer to the Backport tool documentation

achyutjhunjhunwala · 2025-06-19T12:10:15Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.19`: - [[Logs] Replace log stream component (#219425)](#219425)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

achyutjhunjhunwala added 3 commits April 28, 2025 17:38

Replace log stream component on transaction details page

6b3a426

Merge branch 'main' into replace-log-stream-component

6158dbf

Merge branch 'main' into replace-log-stream-component

5193c6b

weltenwort mentioned this pull request May 9, 2025

[Logs UI] Cover the dashboard-embeddable log stream in a functional test #91717

Closed

achyutjhunjhunwala added 2 commits May 26, 2025 10:32

Merge branch 'main' into replace-log-stream-component

d197478

Merge branch 'main' into replace-log-stream-component

0059886

achyutjhunjhunwala self-assigned this May 28, 2025

achyutjhunjhunwala added backport:version Backport to applied version labels v8.19.0 Feature:Discover Discover Application Feature:Logs UI Logs UI feature Team:obs-onboarding Observability Onboarding Team release_note:skip Skip the PR/issue when compiling release notes labels May 28, 2025

achyutjhunjhunwala and others added 17 commits May 28, 2025 18:03

Fix passing long Discover locator inside execution context which was …

a9660b2

…getting truncated

Enable Flyout for all new Logs Saved Search Embeddables

635e477

Replace Log stream component from Host List page

2eaaec5

replace log stream in hosts detail view which in addition also impact…

0e09b27

…s the flyout which can be opened from hosts list view

Merge branch 'main' into replace-log-stream-component

69fc87d

Fix the locator passing error

145ac87

Merge branch 'main' into replace-log-stream-component

e6586c2

[CI] Auto-commit changed files from 'node scripts/styled_components_m…

7d89fbc

…apping'

Add new profile for handling Fleet Logs

1cb50a2

[CI] Auto-commit changed files from 'ts-node .buildkite/pipeline-reso…

08439cf

…urce-definitions/scripts/fix-location-collection.ts'

Fix quick check issues

a4210d3

Replace the newly added fleet profile with custom column logic

6e34237

Add logic to merge all controls which open flyout under 1 flag and up…

1ceac5c

…date LogStream Component in Category page as well

Merge branch 'main' into replace-log-stream-component

fbb66e3

Fix Unit test for agent_logs

bbc425d

Fix initialize edit unit tests

fe79cb6

Fix execution context browser ftr tests

ecf91e7

Add logic to handle log stream dashboards from older versions to load…

58094fb

… with Saved Search

juliaElastic approved these changes Jun 17, 2025

View reviewed changes

Merge branch 'main' into replace-log-stream-component

afd4f42

achyutjhunjhunwala added the v9.1.0 label Jun 17, 2025

tonyghiani approved these changes Jun 17, 2025

View reviewed changes

This was referenced Jun 17, 2025

[One Discover] [Logs] Expand Action Not Opening DocViewer Flyout in Saved Search Embeddables #224183

Closed

[One Discover] [Logs] Remove old Logs Stream codebase #224184

Closed

davismcphee requested changes Jun 18, 2025

View reviewed changes

achyutjhunjhunwala added 3 commits June 18, 2025 11:53

Add logic to new legacy embeddable to handle the Logs Stream embeddab…

466188d

…le not polluting Discover Saved Search embeddable

Merge branch 'main' into replace-log-stream-component

9f1eaa1

Enable search bar in Service Logs tab

5b78f6c

achyutjhunjhunwala requested a review from davismcphee June 18, 2025 12:11

achyutjhunjhunwala added 3 commits June 18, 2025 14:25

Enable search bar in Service Logs tab for Mobile Services

8956519

Fix height for logs tab in infra

77a0656

Revert enabling the search bar in APM

c076e74

davismcphee approved these changes Jun 18, 2025

View reviewed changes

achyutjhunjhunwala added 2 commits June 19, 2025 11:41

Add missing search bar on Hosts page which was deleted by mistake

7dbc569

Merge branch 'main' into replace-log-stream-component

d73255b

achyutjhunjhunwala enabled auto-merge (squash) June 19, 2025 09:43

achyutjhunjhunwala merged commit d9ebb93 into elastic:main Jun 19, 2025
11 checks passed

achyutjhunjhunwala deleted the replace-log-stream-component branch June 19, 2025 11:45

achyutjhunjhunwala mentioned this pull request Jun 19, 2025

[8.19] [Logs] Replace log stream component (#219425) #224558

Merged

kibanamachine mentioned this pull request Jun 19, 2025

Implement functionality to add observables, procedures and custom fields to alerts for TheHive #207255

Merged

7 tasks

kpollich mentioned this pull request Aug 7, 2025

[Fleet] Per-agent log-level setting is missing #230980

Closed

gbamparop mentioned this pull request Jan 13, 2026

[Logs UX] [Obs Exploration] fix incorrect extraction of time range from wrong context #248795

Merged

Conversation

achyutjhunjhunwala commented Apr 28, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's Pending ?

Uh oh!

juliaElastic left a comment

Choose a reason for hiding this comment

Uh oh!

tonyghiani left a comment

Choose a reason for hiding this comment

Uh oh!

davismcphee left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

davismcphee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Jun 19, 2025

Uh oh!

elasticmachine commented Jun 19, 2025

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

async chunk count

History

Uh oh!

kibanamachine commented Jun 19, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

achyutjhunjhunwala commented Jun 19, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

achyutjhunjhunwala commented Apr 28, 2025 •

edited by kibanamachine

Loading