Skip to content

Commit cd0944c

Browse files
committed
[Refactor] Complete metrics overhaul
Metrics got an entire overhaul. Instead of relying on a broken prometheus library to publish our metrics, we now use the `tracing` library and with OpenTelemetry that we bind together then publish into a prometheus library. Metrics are now mostly derive-macros. This means that the struct can express what it wants to export and a help text. The library will choose if it is able to export it. Tracing now works by calling `.publish()` on the parent structs, those structs need to call `.publish()` on all the child members it wishes to publish data about. If a "group" is requested, use the `group!()` macro, which under-the-hood calls `tracing::span` with some special labels. At primitive layers, it will call the `publish!()` macro, which will call `tracing::event!()` macro under-the-hood with some special fields set. A custom `tracing::Subscriber` will intercept all the events and spans and convert them into a json-like object. This object can then be exported as real json or encoded into other formats like otel/prometheus. closes: TraceMachina#1164, TraceMachina#650, TraceMachina#384, TraceMachina#209 towards: TraceMachina#206
1 parent 3574149 commit cd0944c

File tree

80 files changed

+2729
-1490
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+2729
-1490
lines changed

BUILD.bazel

+8
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ rust_binary(
1616
deps = [
1717
"//nativelink-config",
1818
"//nativelink-error",
19+
"//nativelink-metric",
20+
"//nativelink-metric-collector",
1921
"//nativelink-proto",
2022
"//nativelink-scheduler",
2123
"//nativelink-service",
@@ -28,16 +30,22 @@ rust_binary(
2830
"@crates//:futures",
2931
"@crates//:hyper",
3032
"@crates//:mimalloc",
33+
"@crates//:opentelemetry",
34+
"@crates//:opentelemetry-prometheus",
35+
"@crates//:opentelemetry_sdk",
3136
"@crates//:parking_lot",
37+
"@crates//:prometheus",
3238
"@crates//:prometheus-client",
3339
"@crates//:rustls-pemfile",
3440
"@crates//:scopeguard",
41+
"@crates//:serde_json",
3542
"@crates//:serde_json5",
3643
"@crates//:tokio",
3744
"@crates//:tokio-rustls",
3845
"@crates//:tonic",
3946
"@crates//:tower",
4047
"@crates//:tracing",
48+
"@crates//:tracing-subscriber",
4149
],
4250
)
4351

Cargo.lock

+203-7
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)