[Logs+] Implement Logs Data Stream selector#156479
Closed
tonyghiani wants to merge 141 commits intoelastic:mainfrom
Closed
[Logs+] Implement Logs Data Stream selector#156479tonyghiani wants to merge 141 commits intoelastic:mainfrom
tonyghiani wants to merge 141 commits intoelastic:mainfrom
Conversation
- A container to centralize functionality around the usage of saved searches in Discover
….tsx Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com>
…-ref HEAD~1..HEAD --fix'
…UNINITIALIZED is given
… loading the saved search
…-ref HEAD~1..HEAD --fix'
…tonyghiani/kibana into 2655-implement-log-data-stream-selector
…-ref HEAD~1..HEAD --fix'
… integration machine events
…tonyghiani/kibana into 2655-implement-log-data-stream-selector
…-ref HEAD~1..HEAD --fix'
⏳ Build in-progress, with failuresFailed CI Steps
Test Failures
History
To update your PR or re-run it, just comment with: |
Contributor
Author
|
Rebased with #159907 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📓 Summary
Closes
This PR introduces a customized log consumption experience in the Discover plugin. By leveraging the new
observability_logsplugin and utilizing thediscover.customizefunctionality, we have curated a more tailored user experience.The key feature of this implementation is the
DataStreamSelectorcomponent, which replaces the original DiscoverDataViewPicker. It handles the retrieval, rendering, and navigation of integrations and data streams related to logs, providing an improved user interface.This PR involves significant development efforts, including the creation of the
observability_logsplugin, implementation of services, state machines, custom hooks, and enhancements to presentational components. The following overview will help reviewers understand the responsibilities of each component in this implementation.demo-selector-fhd.mov
DataStreamsService & DataStreamsClient
The DataStreamsService is introduced, a crucial component that mediates access to the newly implemented DataStreamsClient. During the plugin's lifecycle, the DataStreamsService exposes a client property through its start() method, providing convenient access to a DataStreamsClient instance.
The DataStreamsClient is responsible for abstracting the data fetching process for two endpoints: the integrations endpoint and the data streams listing endpoint. These endpoints are utilized to populate the selector options in the user interface. To facilitate this, the DataStreamsClient exposes the findIntegrations and findDataStreams methods, which handle the respective data fetching.
Discover Customization
The critical part of this work consists of where the customization is applied.
Inside the
public/plugin.tsx, we lazy load and create, injecting the required dependencies, theCustomDataStreamSelector, which already encapsulates all the logic required to make the selector work with the external APIs.We kept separating the data fetching logic from how the selector works, and all the data and events are passed into the UI component with properties.
Data fetching state machines & custom hooks
To handle the data fetching of integrations and unmanaged data streams, we created two different state machines to separately handle the related action for each dataset, such as remote search, in-memory search, error handling etc.
Integration machine and useIntegrations
The integrations state machine handles automatic data fetching of the resources and additionally provides transitions for loading more integrations, searching integrations by HTTP request, searching locally into integration streams, and all the related loading and error handling states.
It is then interpreted inside the
useIntegrationscustom hook, which exposes the fetched data and handlers for all the above-mentioned actions.Data streams machine and useDataStreams
Similar to the integrations state machine, but simplified since the data streams search can only happen with HTTP requests and there is no pagination that requires to handle the load of more entries.
It is interpreted inside the
useDataStreamscustom hook, which also exposes the fetched data and handlers for the available actions.DataStreamSelector
The
DataStreamSelectorcomponent contains all the logic that manages the navigation and searches across the different panels that render integrations, integrations' streams or unmanaged streams.As the datasets come from different APIs or are performed in-memory, the search work follow this logic:
EuiContextMenu), the search is done with an HTTP request.EuiContextMenu), the search is done in-memory, filtering and sorting directly in the client.EuiContextMenu), the search is done again with an HTTP request.To handle these possible user journeys correctly without side effects, we created another state machine and exposed its actions with an internal
useDataStreamSelectorcustom hook.Next steps
This component will change quite a lot until we won't get to a final design. As soon as a first solid mvp is defined for production, a complete test for the component will be implemented, among with a more generic functional test for the core customization features.