Total disk usage using metrics

[CasperThrane]

Hi

I am scrapping prometheus metrics, using Elastic MetricBeat. I can see i get this value prometheus.metrics.kafka_log_log_size. But is by partion. I don't know if I can use this to calculate the total disk usage. If I try to summerize the partion by topic, I am not getting the same result, as I can see in ex. Kafka UI. Even if I multiply with this prometheus.metrics.kafka_cluster_partition_replicascount, I still get a value which is not the same as the Kafka UI value. Off course Kafka UI could show it in a wrong way.

Br
Casper Thrane

[scholzj]

The Kafka log size is a size of the log per partition. If you want a metric with overall disk usage / free space, the best is to scrape it from Kubernetes. Most Kubernetes distros provide it in one way or another for all PVs. But how to enable it or get it might be specific for your distribution. (I do not know what or how Kafka UI shows, so no idea how it compares to that)

[CasperThrane]

I just wanted the size of each topic, and for that I can us prometheus.metrics.kafka_log_log_size. I found out, the Kafka log size is the same Kafka UI is using. The problem i faced in Kafka UI, is related to rounding of the size value.