Skip to content

Commit

Permalink
[Refactor] Complete metrics overhaul
Browse files Browse the repository at this point in the history
Metrics got an entire overhaul. Instead of relying on a broken
prometheus library to publish our metrics, we now use the
`tracing` library and with OpenTelemetry that we bind together
then publish into a prometheus library.

Metrics are now mostly derive-macros. This means that the struct
can express what it wants to export and a help text. The library
will choose if it is able to export it.

Tracing now works by calling `.publish()` on the parent structs,
those structs need to call `.publish()` on all the child members
it wishes to publish data about. If a "group" is requested, use
the `group!()` macro, which under-the-hood calls `tracing::span`
with some special labels. At primitive layers, it will call the
`publish!()` macro, which will call `tracing::event!()` macro
under-the-hood with some special fields set. A custom
`tracing::Subscriber` will intercept all the events and spans
and convert them into a json-like object. This object can then
be exported as real json or encoded into other formats like
otel/prometheus.

closes: #1164, #650, #384, #209
towards: #206
  • Loading branch information
allada committed Jul 25, 2024
1 parent 3574149 commit fe4c78c
Show file tree
Hide file tree
Showing 80 changed files with 2,675 additions and 1,466 deletions.
8 changes: 8 additions & 0 deletions BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,25 @@ rust_binary(
"//nativelink-store",
"//nativelink-util",
"//nativelink-worker",
"//nativelink-metric",
"//nativelink-metric-collector",
"@crates//:async-lock",
"@crates//:axum",
"@crates//:clap",
"@crates//:opentelemetry",
"@crates//:opentelemetry-prometheus",
"@crates//:opentelemetry_sdk",
"@crates//:tracing-subscriber",
"@crates//:futures",
"@crates//:hyper",
"@crates//:prometheus",
"@crates//:mimalloc",
"@crates//:parking_lot",
"@crates//:prometheus-client",
"@crates//:rustls-pemfile",
"@crates//:scopeguard",
"@crates//:serde_json5",
"@crates//:serde_json",
"@crates//:tokio",
"@crates//:tokio-rustls",
"@crates//:tonic",
Expand Down
Loading

0 comments on commit fe4c78c

Please sign in to comment.