[Logs UI] Fetch single log entries via a search strategy by weltenwort · Pull Request #81710 · elastic/kibana

weltenwort · 2020-10-27T01:12:06Z

Summary

This replaces the log item API with a single-log-entry search strategy. This is used to fetch the data for display in the details flyout. There should be no significant visual difference to the user.

Request cancellation support will be added in a follow-up PR.

closes #78001

Implementation details

I've commented inline on various aspects of the implementation.

The search strategy internally delegates to the ese search strategy to fetch the actual data.
This introduces a server-side service directory structure to match the platform conventions regarding plugin substructure:
- a LogEntrySearchStrategy
- a LogEntriesService, which registers the above search strategy
The typed_search_strategy.ts utilities are designed to be re-usable in upcoming search-strategy-related PRs.

Previews

Follow-up tasks

Cancel request when component unmounts while a full response is pending ([Logs UI] Add helper hooks with search strategy request cancellation #83906)
Move the filter button to an action column and ensure it works for array values

afgomez

👀 I did a first pass on the code. I like it overall! It has taught me a lot about RxJS.

I found an issue where loading an entry fails silently, so there might be a problem in how errors are handled. I'll dig a bit more into it tomorrow.

afgomez · 2020-11-24T14:58:22Z

x-pack/plugins/infra/server/services/log_entries/types.ts

+ * you may not use this file except in compliance with the Elastic License.
+ */
+
+// import type { CoreSetup, CoreStart } from 'kibana/public';


I think this can be removed ✂️

x-pack/plugins/infra/server/services/log_entries/log_entries_service.ts

x-pack/plugins/infra/server/services/log_entries/queries/log_entry.ts

x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.ts

x-pack/plugins/infra/server/utils/typed_search_strategy.ts

x-pack/plugins/infra/server/services/log_entries/queries/log_entry.ts

afgomez

I did a more functional pass today. Overall it works as intended. However I found two things.

The first is what I commented on yesterday. When using the dev cluster it tries to load indefinitely, then it fails silently. The error seems to be on the ES side, but it's not being handled in the UI.

The second: the fields are no longer displayed shorted.

Before:

After:

weltenwort · 2020-11-26T11:45:14Z

Thank you for the review ❤️

When using the dev cluster it tries to load indefinitely, then it fails silently. The error seems to be on the ES side, but it's not being handled in the UI.

This is caused by the fields: ['*'] bug, that you noticed a few weeks ago. It's fixed by now, but the cluster is stuck on an old version. I developed against a local cluster.

the fields are no longer displayed shorted

Good catch, I'll fix that.

weltenwort · 2020-11-26T17:29:17Z

@afgomez, I replaced the EuiBasicTable with an EuiInMemoryTable, which gives us sorting without extra effort. Because it was so easy I also enabled the search box above the table so the fields can be filtered interactively:

I'd like to tackle the error display in #83906, because it completely replaces the current client-side code that calls the search strategy. Would that work for you?

afgomez · 2020-12-02T14:32:44Z

I replaced the EuiBasicTable with an EuiInMemoryTable, which gives us sorting. [...] I also enabled the search box above the table so the fields can be filtered interactively.

Nice! I tried the filter and it's very useful :)

I noticed there are two fields missing: _id and _index.

We added them manually in the API. I see the _index is also present as index but there's no corresponding id field.

Maybe it makes more sense to show the ID somewhere else in the UI, like in the header:

afgomez · 2020-12-02T14:34:02Z

When using the dev cluster it tries to load indefinitely, then it fails silently. The error seems to be on the ES side, but it's not being handled in the UI.

This is caused by the fields: ['*'] bug, that you noticed a few weeks ago.

My comment was more related to the UI not handling the error. I'm not sure we do any error handling in master either, so maybe we can tackle that on a separate PR.

afgomez

Everything works 🎉! There is the question of what to do with the added _id and _index fields. I'm not explicitly approving to keep track of that, but consider it virtually approved :)

weltenwort · 2020-12-02T17:30:09Z

Maybe it makes more sense to show the ID somewhere

Good point, I forgot about rendering id and index. I'll try to find a place to display them in the flyout. 👍

The errors were indeed silent even before this change. I'm cleaning that up in #83906, which will be ready for review soon.

weltenwort · 2020-12-02T18:15:41Z

@afgomez I think the header works reasonably well:

What do you think?

afgomez · 2020-12-03T09:36:14Z

x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_flyout.tsx

+                  defaultMessage="Details for log entry {logEntryId}"
                  id="xpack.infra.logFlyout.flyoutTitle"
+                  values={{
+                    logEntryId: flyoutItem ? <code>{flyoutItem.id}</code> : '',


Shall we use <EuiCode> here?

I considered that, but decided against it, because <EuiCode> is a pretty complex component that performs language-specific syntax highlighting on the content. It felt like overkill here, since we're not displaying code but only want to stylize the immutable values.

afgomez

I think the header works reasonably well

Yeah I think so too :) I would consider using <EuiCode> instead of <code>, but other than that LGTM!

kibanamachine · 2020-12-03T11:59:47Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 2fa8a93

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`infra`	1051	1054	+3

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`infra`	2.6MB	2.6MB	+4.2KB

Distributable file count

id	before	after	diff
`default`	43460	43466	+6

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`infra`	181.0KB	182.1KB	+1.1KB

History

💚 Build #91478 succeeded ff57b2e
💔 Build #91471 failed 997fee0
💚 Build #90656 succeeded c80de2f
💚 Build #90473 succeeded f87b248
💔 Build #90438 failed c4e6e32

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…overy-action-group * upstream/master: (48 commits) [Lens] accessibility screen reader issues (elastic#84395) [Logs UI] Fetch single log entries via a search strategy (elastic#81710) fix: 🐛 don't add separator befor group on no main items (elastic#83166) [Security Solution][Detections] Implements indicator match rule cypress test (elastic#84323) [APM] Add APM agent config options (elastic#84678) Fixed a11y issue on rollup jobs table selection (elastic#84567) [Discover] Refactor getContextUrl to separate file (elastic#84503) [Embeddable] Export CSV action for Lens embeddables in dashboard (elastic#83654) [TSVB] [Cleanup] Remove extra dateFormat props (elastic#84749) [Lens] Migrate legacy es client and remove total hits as int (elastic#84340) Improve logging pipeline in @kbn/legacy-logging (elastic#84629) Catch @hapi/podium errors (elastic#84575) [Discover] Unskip date histogram test (elastic#84727) Rename server.xsrf.whitelist to server.xsrf.allowlist (elastic#84791) [Enterprise Search] Fix schema errors button (elastic#84842) [APM] Removes react-sticky dependency in favor of using CSS (elastic#84589) [Maps] Always initialize routes on server-startup (elastic#84806) [Fleet] EPM support to handle uploaded file paths (elastic#84708) [Snapshot Restore] Fix initial policy form state (elastic#83928) Upgrade Node.js to version 14 (elastic#83425) ...

… (#84890) Backports the following commits to 7.x: - [Logs UI] Fetch single log entries via a search strategy (#81710)

add empty log entry search strategy

1982fd4

weltenwort added v8.0.0 Feature:Logs UI Logs UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v7.11.0 labels Oct 27, 2020

weltenwort added this to the Logs UI 7.11 milestone Oct 27, 2020

weltenwort self-assigned this Oct 27, 2020

weltenwort added 8 commits October 27, 2020 19:23

WIP: send query to ES and parse response

896656c

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

51c8458

WIP: use the source config to extract fields

ea1eb75

Pass sources dependency

9e65553

Fix LogEntry imports

79e555a

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

a6ceba1

Fix source config fetching and state encoding

b48e5de

Add basic unit test

edd4047

afgomez self-requested a review November 3, 2020 10:48

weltenwort added 14 commits November 10, 2020 15:08

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

6a07bec

Adapt to changed search strategy signature

7d9cc2e

Add tests and fix bugs

78a6bef

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

8bb0d80

Wire up client-side search strategy usage

2327000

Make validation error messages more readable

d1c7b2d

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

0a43fb5

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

05f8212

Fix types and props

7dac882

Factor out async request runtime types

b402407

Remove superfluous take

99a8d64

Factor out search strategy error utilities

737ec78

Satisfy linter

856ae62

Reintroduce take(1) and ensure branches are exclusive

ca7681a

afgomez reviewed Nov 24, 2020

View reviewed changes

afgomez suggested changes Nov 25, 2020

View reviewed changes

weltenwort added 5 commits November 26, 2020 13:14

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

b24c3a6

Replace EuiBasicTable with EuiInMemoryTable for sorting

c4e6e32

Work around type error due to licensing ambient type

3fa56e0

Remove commented-out code

8076833

Rename get/submit to recovered/initial

f87b248

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

c80de2f

weltenwort requested a review from afgomez November 30, 2020 12:05

afgomez reviewed Dec 2, 2020

View reviewed changes

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

fb3b2b6

Add log entry id and index to flyout header

997fee0

weltenwort requested a review from afgomez December 2, 2020 18:19

Update translations

ff57b2e

afgomez reviewed Dec 3, 2020

View reviewed changes

afgomez approved these changes Dec 3, 2020

View reviewed changes

Merge branch 'master' into logs-ui-add-log-entry-search-strategy

2fa8a93

weltenwort merged commit 3cb26eb into elastic:master Dec 3, 2020

weltenwort deleted the logs-ui-add-log-entry-search-strategy branch December 3, 2020 12:05

weltenwort mentioned this pull request Dec 3, 2020

[7.x] [Logs UI] Fetch single log entries via a search strategy (#81710) #84890

Merged

weltenwort added a commit that referenced this pull request Dec 3, 2020

[7.x] [Logs UI] Fetch single log entries via a search strategy (#81710)…

6f9c96b

… (#84890) Backports the following commits to 7.x: - [Logs UI] Fetch single log entries via a search strategy (#81710)

weltenwort mentioned this pull request Jun 2, 2021

[Logs UI] Show warning when CCS searches omit data due to version mismatch #101130

Closed

Conversation

weltenwort commented Oct 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Implementation details

Previews

Follow-up tasks

Uh oh!

afgomez left a comment

Choose a reason for hiding this comment

Uh oh!

afgomez Nov 24, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

afgomez left a comment

Choose a reason for hiding this comment

Uh oh!

weltenwort commented Nov 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weltenwort commented Nov 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

afgomez commented Dec 2, 2020

Uh oh!

afgomez commented Dec 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

afgomez left a comment

Choose a reason for hiding this comment

Uh oh!

weltenwort commented Dec 2, 2020

Uh oh!

weltenwort commented Dec 2, 2020

Uh oh!

afgomez Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

weltenwort Dec 3, 2020

Choose a reason for hiding this comment

Uh oh!

afgomez left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Dec 3, 2020

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Distributable file count

Page load bundle

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

weltenwort commented Oct 27, 2020 •

edited

Loading

weltenwort commented Nov 26, 2020 •

edited

Loading

weltenwort commented Nov 26, 2020 •

edited

Loading

afgomez commented Dec 2, 2020 •

edited

Loading