[streams] lifecycle - ingestion and total docs metadata#210301
Merged
klacabane merged 34 commits intoelastic:mainfrom Feb 17, 2025
Merged
[streams] lifecycle - ingestion and total docs metadata#210301klacabane merged 34 commits intoelastic:mainfrom
klacabane merged 34 commits intoelastic:mainfrom
Conversation
klacabane
added
release_note:skip
klacabane
commented
Feb 11, 2025
Comment on lines
+48
to
+49
| t.union([t.type({ types: typesRt }), t.type({ datasetQuery: t.string })]), | ||
| t.partial({ includeCreationDate: toBooleanRt }), |
Contributor
Author
There was a problem hiding this comment.
We either use types or datasetQuery in the route. Updated the type accordingly
klacabane
commented
Feb 11, 2025
.../plugins/shared/dataset_quality/server/routes/data_streams/get_data_streams_creation_date.ts
Outdated
Show resolved
Hide resolved
tonyghiani
reviewed
Feb 14, 2025
...lugins/streams_app/public/components/stream_detail_lifecycle/hooks/use_data_stream_stats.tsx
Outdated
Show resolved
Hide resolved
Contributor
Bootstrap fixed it, as I mentioned above another issue is it get stuck on the loading state for any error that might occur. |
Contributor
Author
|
@yngrdyn @tonyghiani should be ready for another review |
Contributor
Author
|
9f1357f updates the datepicker to use the one already defined throughout the app which shares the range already defined. We can revisit the default in lifecycle table if that does not make sense |
Contributor
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Public APIs missing exports
Page load bundle
Unknown metric groupsAPI count
ESLint disabled line counts
Total ESLint disabled count
History
|
tonyghiani
approved these changes
Feb 17, 2025
Contributor
tonyghiani
left a comment
tonyghiani
left a comment
There was a problem hiding this comment.
Looks good, thanks for the changes
Contributor
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/13370377328 |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added
the
backport missing
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
klacabane
added a commit
to klacabane/kibana
that referenced
this pull request
Feb 20, 2025
Adds avg ingestion per day, total doc count and ingestion rate graph to the lifecycle view. We use the dataset quality plugin to compute these values. I've added a query string to optionally retrieve the creation date of a data stream in the `data_streams/stats` endpoint.  ----- @elastic/obs-ux-logs-team the change in dataset quality involves the optional retrieval of the data streams creation date in the `/stats` endpoint. There are other ways in dataset quality to get these informations but they rely on queries to compute the data. In our case these queries will always be unbounded and using the `/stats` would be more efficient as it relies on cluster state. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 95b3f6e)
Contributor
|
Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync. |
klacabane
added a commit
that referenced
this pull request
Feb 20, 2025
… (#211875) # Backport This will backport the following commits from `main` to `8.x`: - [[streams] lifecycle - ingestion and total docs metadata (#210301)](#210301) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Kevin Lacabane","email":"kevin.lacabane@elastic.co"},"sourceCommit":{"committedDate":"2025-02-17T12:35:20Z","message":"[streams] lifecycle - ingestion and total docs metadata (#210301)\n\nAdds avg ingestion per day, total doc count and ingestion rate graph to\nthe lifecycle view.\n\nWe use the dataset quality plugin to compute these values. I've added a\nquery string to optionally retrieve the creation date of a data stream\nin the `data_streams/stats` endpoint.\n\n\n\n-----\n\n@elastic/obs-ux-logs-team the change in dataset quality involves the\noptional retrieval of the data streams creation date in the `/stats`\nendpoint. There are other ways in dataset quality to get these\ninformations but they rely on queries to compute the data. In our case\nthese queries will always be unbounded and using the `/stats` would be\nmore efficient as it relies on cluster state.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"95b3f6e14da782208dc701c46e7c8bbd77cc55e1","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport missing","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[streams] lifecycle - ingestion and total docs metadata","number":210301,"url":"https://github.com/elastic/kibana/pull/210301","mergeCommit":{"message":"[streams] lifecycle - ingestion and total docs metadata (#210301)\n\nAdds avg ingestion per day, total doc count and ingestion rate graph to\nthe lifecycle view.\n\nWe use the dataset quality plugin to compute these values. I've added a\nquery string to optionally retrieve the creation date of a data stream\nin the `data_streams/stats` endpoint.\n\n\n\n-----\n\n@elastic/obs-ux-logs-team the change in dataset quality involves the\noptional retrieval of the data streams creation date in the `/stats`\nendpoint. There are other ways in dataset quality to get these\ninformations but they rely on queries to compute the data. In our case\nthese queries will always be unbounded and using the `/stats` would be\nmore efficient as it relies on cluster state.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"95b3f6e14da782208dc701c46e7c8bbd77cc55e1"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210301","number":210301,"mergeCommit":{"message":"[streams] lifecycle - ingestion and total docs metadata (#210301)\n\nAdds avg ingestion per day, total doc count and ingestion rate graph to\nthe lifecycle view.\n\nWe use the dataset quality plugin to compute these values. I've added a\nquery string to optionally retrieve the creation date of a data stream\nin the `data_streams/stats` endpoint.\n\n\n\n-----\n\n@elastic/obs-ux-logs-team the change in dataset quality involves the\noptional retrieval of the data streams creation date in the `/stats`\nendpoint. There are other ways in dataset quality to get these\ninformations but they rely on queries to compute the data. In our case\nthese queries will always be unbounded and using the `/stats` would be\nmore efficient as it relies on cluster state.\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"95b3f6e14da782208dc701c46e7c8bbd77cc55e1"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
kibanamachine
removed
the
backport missing
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Mar 22, 2025
Adds avg ingestion per day, total doc count and ingestion rate graph to the lifecycle view. We use the dataset quality plugin to compute these values. I've added a query string to optionally retrieve the creation date of a data stream in the `data_streams/stats` endpoint.  ----- @elastic/obs-ux-logs-team the change in dataset quality involves the optional retrieval of the data streams creation date in the `/stats` endpoint. There are other ways in dataset quality to get these informations but they rely on queries to compute the data. In our case these queries will always be unbounded and using the `/stats` would be more efficient as it relies on cluster state. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds avg ingestion per day, total doc count and ingestion rate graph to the lifecycle view.
We use the dataset quality plugin to compute these values. I've added a query string to optionally retrieve the creation date of a data stream in the
data_streams/statsendpoint.@elastic/obs-ux-logs-team the change in dataset quality involves the optional retrieval of the data streams creation date in the
/statsendpoint. There are other ways in dataset quality to get these informations but they rely on queries to compute the data. In our case these queries will always be unbounded and using the/statswould be more efficient as it relies on cluster state.