Skip to content

[DSL] Introduce new endpoint to expose data stream lifecycle stats#101845

Merged
gmarouli merged 16 commits intoelastic:mainfrom
gmarouli:data-stream-lifecycle-stats
Nov 20, 2023
Merged

[DSL] Introduce new endpoint to expose data stream lifecycle stats#101845
gmarouli merged 16 commits intoelastic:mainfrom
gmarouli:data-stream-lifecycle-stats

Conversation

@gmarouli
Copy link
Contributor

@gmarouli gmarouli commented Nov 6, 2023

We introduce the endpoint GET _lifecycle/stats that exposes stats from the data stream lifecycle run and the data streams and backing indices managed by DSL. For example:

GET _lifecycle/stats?human&pretty
{
  "last_run_duration_in_millis": 2,
  "last_run_duration": "2ms",
  "time_between_starts_in_millis": 9998,
  "time_between_starts": "9.99s",
  "data_streams_count": 2,
  "data_streams": [
    {
      "name": "my-other-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 1
    },
    {
      "name": "my-data-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 0
    }
  ]
}

@gmarouli gmarouli added the :StorageEngine/Data streams Data streams and their lifecycles label Nov 6, 2023
@gmarouli
Copy link
Contributor Author

gmarouli commented Nov 9, 2023

@elasticmachine update branch

});
}

private static List<String> getBackingIndices(String dataStreamName) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved this helper function in the bottom to be after all the test cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this note, this is really helpful when reviewing ❤️

@gmarouli
Copy link
Contributor Author

@elasticmachine update branch

@gmarouli gmarouli requested a review from andreidan November 15, 2023 10:03
@gmarouli gmarouli marked this pull request as ready for review November 15, 2023 10:03
@elasticsearchmachine elasticsearchmachine added the Team:Data Management (obsolete) DO NOT USE. This team no longer exists. label Nov 15, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @gmarouli, I've created a changelog YAML for you.

@gmarouli gmarouli changed the title Introduce new endpoint to expose data stream lifecycle stats [DSL] Introduce new endpoint to expose data stream lifecycle stats Nov 15, 2023
Copy link
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this Mary, this is really nice ! 🚀

LGTM

@@ -0,0 +1,93 @@
[[data-streams-get-lifecycle-stats]]
=== Get data stream lifecycle stats
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice ! ❤️

});
}

private static List<String> getBackingIndices(String dataStreamName) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this note, this is really helpful when reviewing ❤️

}
}

public static class Response extends ActionResponse implements ChunkedToXContentObject {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chunked response ❤️

@gmarouli
Copy link
Contributor Author

@elasticmachine update branch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :StorageEngine/Data streams Data streams and their lifecycles Team:Data Management (obsolete) DO NOT USE. This team no longer exists. v8.12.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments