[metrics] Report resources allocated to Ray for Grafana dashboard

[kathryn-zhou]

Describe your feature request

PR #14184 removed deprecated code paths that included Local Available Resource metrics that reported how much of each resource was allocated to Ray. Could we have the new scheduler expose metrics like this so that we can export to Prometheus and then Grafana?

For feature requests or questions, post on our Discussion page instead: https://discuss.ray.io/

[simon-mo]

@wuisawesome, @ericl, can you give some pointers about where would be the best place to add them? @kathryn-zhou can take a stab at it.

[ericl]

@edoakes @rkooo567 what's the state of metrics reporting? Should the raylet emit metrics via OpenCensus? Should the GCS do it instead? Or are there other possibilities here?

[edoakes]

@ericl both raylet and the GCS can emit metrics via OpenCensus and it'll show up in the prometheus endpoints. I'm not opinionated about which should do this, but it's probably more useful to have per-node information so maybe raylet is the natural place.

[ericl]

Sounds good, so it seems like we should be emitting this from raylet via the existing OpenCensus metrics.

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[stale]

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

• If you'd like to keep the issue open, just leave any comment, and the stale label will be removed!
• If you'd like to get more attention to the issue, please tag one of Ray's contributors.

You can always ask for help on our discussion forum or Ray's public slack channel.

[stale]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you'd still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for opening the issue!