Skip to content

[8.x] [streams] lifecycle - ingestion and total docs metadata (#210301)#211875

Merged
klacabane merged 2 commits intoelastic:8.xfrom
klacabane:backport/8.x/pr-210301
Feb 20, 2025
Merged

[8.x] [streams] lifecycle - ingestion and total docs metadata (#210301)#211875
klacabane merged 2 commits intoelastic:8.xfrom
klacabane:backport/8.x/pr-210301

Conversation

@klacabane
Copy link
Copy Markdown
Contributor

Backport

This will backport the following commits from main to 8.x:

Questions ?

Please refer to the Backport tool documentation

Adds avg ingestion per day, total doc count and ingestion rate graph to
the lifecycle view.

We use the dataset quality plugin to compute these values. I've added a
query string to optionally retrieve the creation date of a data stream
in the `data_streams/stats` endpoint.

![Screenshot 2025-02-11 at 17 39
13](https://github.com/user-attachments/assets/9242ecbc-ebee-43da-b742-fbc0d0997bc2)

-----

@elastic/obs-ux-logs-team the change in dataset quality involves the
optional retrieval of the data streams creation date in the `/stats`
endpoint. There are other ways in dataset quality to get these
informations but they rely on queries to compute the data. In our case
these queries will always be unbounded and using the `/stats` would be
more efficient as it relies on cluster state.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 95b3f6e)
@klacabane klacabane added the backport This PR is a backport of another PR label Feb 20, 2025
@klacabane klacabane enabled auto-merge (squash) February 20, 2025 09:54
@klacabane klacabane merged commit b846cc9 into elastic:8.x Feb 20, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
streamsApp 301 305 +4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
datasetQuality 14 31 +17

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
datasetQuality 240.8KB 240.8KB +41.0B
streamsApp 289.5KB 293.1KB +3.6KB
total +3.7KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
datasetQuality 8 11 +3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
datasetQuality 19.9KB 20.0KB +52.0B
streamsApp 6.8KB 7.1KB +302.0B
total +354.0B
Unknown metric groups

API count

id before after diff
datasetQuality 14 31 +17

ESLint disabled line counts

id before after diff
streamsApp 8 7 -1

Total ESLint disabled count

id before after diff
streamsApp 12 11 -1

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants