Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metric stdout exporter to print timestamp in human readable format #1192

Merged

Conversation

cijothomas
Copy link
Member

@cijothomas cijothomas commented Aug 8, 2023

Addresses #1191 for metrics.

Changes

Update stdout metric exporter to write timestamp in human readable format.
Writes timestamp in human-friendly format as well. The existing unix_ns format is kept as-is.

Example outout with this PR:
"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404"

{"resourceMetrics":{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"metrics-basic-example"}}]},"scopeMetrics":[{"scope":{"name":"mylibraryname"},"metrics":[{"name":"my_counter","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404","value":10}],"aggregationTemporality":2,"isMonotonic":true}},{"name":"my_observable_counter","description":"My observable counter example description","unit":"myunit","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404","value":100}],"aggregationTemporality":2,"isMonotonic":true}},{"name":"my_updown_counter","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404","value":-10}],"aggregationTemporality":2,"isMonotonic":false}},{"name":"my_observable_updown_counter","description":"My observable updown counter example description","unit":"myunit","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404","value":100}],"aggregationTemporality":2,"isMonotonic":false}},{"name":"my_histogram","description":"My histogram example description","histogram":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":"2023-08-08 00:02:41.404","time":"2023-08-08 00:02:41.404","count":1,"explicitBounds":[0.0,5.0,10.0,25.0,50.0,75.0,100.0,250.0,500.0,750.0,1000.0,2500.0,5000.0,7500.0,10000.0],"bucketCounts":[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0],"min":10.5,"max":10.5,"sum":10.5,"exemplars":[],"flags":0}],"aggregationTemporality":2}},{"name":"my_gauge","description":"A gauge set to 1.0","unit":"myunit","gauge":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTime":null,"time":"2023-08-08 00:02:41.404","value":1.0}]}}]}]}}

before this PR:
"startTimeUnixNano":1691453709811505677,"timeUnixNano":1691453709811786077

{"resourceMetrics":{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"metrics-basic-example"}}]},"scopeMetrics":[{"scope":{"name":"mylibraryname"},"metrics":[{"name":"my_counter","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":1691453709811505677,"timeUnixNano":1691453709811786077,"value":10}],"aggregationTemporality":2,"isMonotonic":true}},{"name":"my_observable_counter","description":"My observable counter example description","unit":"myunit","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":1691453709811553801,"timeUnixNano":1691453709811792600,"value":100}],"aggregationTemporality":2,"isMonotonic":true}},{"name":"my_updown_counter","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":1691453709811602840,"timeUnixNano":1691453709811796162,"value":-10}],"aggregationTemporality":2,"isMonotonic":false}},{"name":"my_observable_updown_counter","description":"My observable updown counter example description","unit":"myunit","sum":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":1691453709811621802,"timeUnixNano":1691453709811798675,"value":100}],"aggregationTemporality":2,"isMonotonic":false}},{"name":"my_histogram","description":"My histogram example description","histogram":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":1691453709811643797,"timeUnixNano":1691453709811801235,"count":1,"explicitBounds":[0.0,5.0,10.0,25.0,50.0,75.0,100.0,250.0,500.0,750.0,1000.0,2500.0,5000.0,7500.0,10000.0],"bucketCounts":[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0],"min":10.5,"max":10.5,"sum":10.5,"exemplars":[],"flags":0}],"aggregationTemporality":2}},{"name":"my_gauge","description":"A gauge set to 1.0","unit":"myunit","gauge":{"dataPoints":[{"attributes":{"mykey1":{"stringValue":"myvalue1"},"mykey2":{"stringValue":"myvalue2"}},"startTimeUnixNano":null,"timeUnixNano":1691453709811778357,"value":1.0}]}}]}]}}

Merge requirement checklist

  • CONTRIBUTING guidelines followed
  • Unit tests added/updated (if applicable)
  • Appropriate CHANGELOG.md files updated for non-trivial, user-facing changes
  • Changes in public API reviewed (if applicable)

@cijothomas cijothomas requested a review from a team August 8, 2023 00:12
@codecov
Copy link

codecov bot commented Aug 8, 2023

Codecov Report

Patch coverage: 23.4% and no project coverage change.

Comparison is base (7607ad3) 49.6% compared to head (1472f58) 49.7%.
Report is 5 commits behind head on main.

Additional details and impacted files
@@          Coverage Diff          @@
##            main   #1192   +/-   ##
=====================================
  Coverage   49.6%   49.7%           
=====================================
  Files        164     172    +8     
  Lines      20653   20713   +60     
=====================================
+ Hits       10258   10304   +46     
- Misses     10395   10409   +14     
Files Changed Coverage Δ
opentelemetry-otlp/src/exporter/grpcio/logs.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/grpcio/mod.rs 35.4% <0.0%> (ø)
opentelemetry-otlp/src/exporter/grpcio/trace.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/http/logs.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/http/metrics.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/http/mod.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/http/trace.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/tonic/logs.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/exporter/tonic/metrics.rs 0.0% <0.0%> (ø)
opentelemetry-otlp/src/lib.rs 32.2% <ø> (ø)
... and 12 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jtescher
Copy link
Member

jtescher commented Aug 8, 2023

Hm maybe this should be based on features? the convenient thing about the current output is it matches the file-exporter, and otlp protobuf json encodings so the common case json format is what you may expect.

@cijothomas
Copy link
Member Author

Hm maybe this should be based on features? the convenient thing about the current output is it matches the file-exporter, and otlp protobuf json encodings so the common case json format is what you may expect.

The file exporter and the json/protobuf are intended for machine consumption, so it makes sense for them to follow the specified format. The stdout exporter is purely for users to learn/debug OpenTelemetry, and not meant for production use. This stdout exporter is more like the "log" exporter in OTel Collector: https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/loggingexporter/internal/otlptext/traces.go#L42

I will propose the following:

  1. Make it clear that stdout is intended only for educational purposes, and there is no guarantee that the output format will remain same overtime. (Also emit a warning note in the stdout exporter so folks will see it, as suggested during SIG call on 8/8/23).
  2. Make a OTLP Exporter (new one, or enhance existing ones), which can send data to file or stdout, following the linked specs. This may be used in Production and the output shape follows the spec and is predictable.

Thoughts on above?

@lalitb
Copy link
Member

lalitb commented Aug 8, 2023

+ 1 for documenting the purpose of stdout clearly.

Another thought would be, as stdout exporter is purely for users to learn/debug OpenTelemetry, and not meant for production use - it allows us to extend it for better debugging. So we can also keep the exiting epoch representation, and add new field for human readable format. Sometime, for debugging purpose, customers may want to configure both stdout, and production exporter and compare the contents to validate the production exporter export values. Or else, keeping it on feature based as @jtescher suggested also works.

@jtescher
Copy link
Member

jtescher commented Aug 8, 2023

Yeah agree that it's about debugging (mostly local) and not production. What I've seen people use it for most is debugging other exports or other integration issues, in which case I think it's mostly around having a second exporter to compare what you expect to see, in which case a json format for what you'd most expect the data to look like for machines and a human readable version like what go has which is more multi-line log oriented and likely not json at all both make sense to me.

@cijothomas
Copy link
Member Author

Thanks for the feedbacks. I do see some value in keeping the unix_nano format.
I will modify the PR so that ts is emitted in both machine and human-friendly formats. I think that'll be a good balance for now.

@cijothomas
Copy link
Member Author

@jtescher modified the PR to keep the existing format, and added user-friendly time as an additional field. Could you review/merge?

Copy link
Member

@jtescher jtescher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good thanks @cijothomas

@jtescher jtescher merged commit 2d42766 into open-telemetry:main Aug 11, 2023
11 of 13 checks passed
@cijothomas cijothomas deleted the cijothomas/stdout-td-humanformat branch August 11, 2023 18:10
edmorley added a commit to edmorley/opentelemetry-rust that referenced this pull request Feb 23, 2024
In open-telemetry#1192, `chrono` was added as a dependency of the `opentelemetry-stdout`
crate in order to support outputting timestamps in human readable format.

In that PR, all Chrono features were disabled apart from the `clock` feature.

However, since that change landed, `chrono` has added support for an even
finer-grained feature named `now`, which is a subset of the `clock` feature which
excludes timezone support, and so avoids pulling in many timezone related crates.

`opentelemetry-stdout` only uses chrono's UTC features, so we can switch
from using the `clock` feature to using `now` instead.

After this change, the following transitive dependencies are no longer pulled in:

- `android-tzdata`
- `android_system_properties`
- `cc`
- `core-foundation-sys`
- `iana-time-zone`
- `iana-time-zone-haiku`
- `windows-core`
- `windows-targets`
- `windows_aarch64_gnullvm`
- `windows_aarch64_msvc`
- `windows_i686_gnu`
- `windows_i686_msvc`
- `windows_x86_64_gnu`
- `windows_x86_64_gnullvm`
- `windows_x86_64_msvc`

See:
chronotope/chrono#1343
https://github.com/chronotope/chrono/blob/main/README.md#crate-features
edmorley added a commit to edmorley/opentelemetry-rust that referenced this pull request Feb 23, 2024
In open-telemetry#1192, `chrono` was added as a dependency of the `opentelemetry-stdout`
crate in order to support outputting timestamps in human readable format.

In that PR, all Chrono features were disabled apart from the `clock` feature.

However, since that change landed, `chrono` has added support for an even
finer-grained feature named `now`, which is a subset of the `clock` feature which
excludes timezone support, and so avoids pulling in many timezone related crates.

`opentelemetry-stdout` only uses chrono's UTC features, so we can switch
from using the `clock` feature to using `now` instead.

After this change, the following transitive dependencies are no longer pulled in:

- `android-tzdata`
- `android_system_properties`
- `cc`
- `core-foundation-sys`
- `iana-time-zone`
- `iana-time-zone-haiku`
- `windows-core`
- `windows-targets`
- `windows_aarch64_gnullvm`
- `windows_aarch64_msvc`
- `windows_i686_gnu`
- `windows_i686_msvc`
- `windows_x86_64_gnu`
- `windows_x86_64_gnullvm`
- `windows_x86_64_msvc`

See:
chronotope/chrono#1343
https://github.com/chronotope/chrono/blob/main/README.md#crate-features
hdost pushed a commit that referenced this pull request Feb 24, 2024
In #1192, chrono was added as a dependency of the opentelemetry-stdout crate in order to support outputting timestamps in human readable format.

In that PR, all Chrono features were disabled apart from the clock feature.

However, since that change landed, chrono v0.4.32 has added support for an even finer-grained feature named now, which is a subset of the clock feature - that excludes timezone support, and so avoids pulling in many timezone related crates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants