Skip to content

feat: add self-observability metrics to otlpmetricgrpc metric exporters#7120

Closed
minimAluminiumalism wants to merge 15 commits intoopen-telemetry:mainfrom
minimAluminiumalism:main
Closed

feat: add self-observability metrics to otlpmetricgrpc metric exporters#7120
minimAluminiumalism wants to merge 15 commits intoopen-telemetry:mainfrom
minimAluminiumalism:main

Conversation

@minimAluminiumalism
Copy link
Copy Markdown

@minimAluminiumalism minimAluminiumalism commented Aug 3, 2025

Fixes #7012

  • Added internal/selfobservability package to gRPC exporters
  • Implemented three metrics as per OpenTelemetry SDK metrics specification:
    • otel.sdk.exporter.metric_data_point.inflight (UpDownCounter)
    • otel.sdk.exporter.metric_data_point.exported (Counter)
    • otel.sdk.exporter.operation.duration (Histogram)

Comment thread internal/shared/otlp/otlpmetric/selfobservability/selfobservability.go.tmpl Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 4, 2025

Codecov Report

❌ Patch coverage is 92.10526% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.9%. Comparing base (c09be18) to head (894c26c).
⚠️ Report is 709 commits behind head on main.

Files with missing lines Patch % Lines
...pc/internal/selfobservability/selfobservability.go 89.3% 9 Missing and 3 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff           @@
##            main   #7120    +/-   ##
======================================
  Coverage   82.9%   82.9%            
======================================
  Files        264     266     +2     
  Lines      24628   24779   +151     
======================================
+ Hits       20425   20561   +136     
- Misses      3820    3832    +12     
- Partials     383     386     +3     
Files with missing lines Coverage Δ
...porters/otlp/otlpmetric/otlpmetricgrpc/exporter.go 85.8% <100.0%> (+2.8%) ⬆️
...ers/otlp/otlpmetric/otlpmetricgrpc/internal/x/x.go 100.0% <100.0%> (ø)
...pc/internal/selfobservability/selfobservability.go 89.3% <89.3%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@minimAluminiumalism
Copy link
Copy Markdown
Author

@pellared I've tried to fix the CI pipeline failures, please review this PR and re-trigger the pipeline check.

@pellared
Copy link
Copy Markdown
Member

pellared commented Aug 4, 2025

@minimAluminiumalism, can you please make the documentation and changelog entry similar to other modules? You can take a look at #7121 for another reference.

@minimAluminiumalism
Copy link
Copy Markdown
Author

@minimAluminiumalism, can you please make the documentation and changelog entry similar to other modules? You can take a look at #7121 for another reference.

The changelog and doc.go have been modified.

Comment thread internal/shared/otlp/otlpmetric/selfobservability/selfobservability.go.tmpl Outdated
Comment thread sdk/internal/x/x.go Outdated
Comment thread internal/shared/otlp/otlpmetric/selfobservability/selfobservability.go.tmpl Outdated
Comment thread exporters/otlp/otlpmetric/otlpmetricgrpc/doc.go Outdated
Comment thread exporters/otlp/otlpmetric/otlpmetricgrpc/exporter.go Outdated
Comment thread exporters/otlp/otlpmetric/otlpmetrichttp/exporter.go Outdated
@flc1125
Copy link
Copy Markdown
Member

flc1125 commented Aug 6, 2025

Since this PR includes 2 components, I suggest implementing one of them first, as it will be easier to get through during both implementation and review.

Comment thread exporters/otlp/otlpmetric/otlpmetrichttp/exporter.go Outdated
Comment thread sdk/internal/x/x.go Outdated
@flc1125
Copy link
Copy Markdown
Member

flc1125 commented Aug 6, 2025

As a supplementary note (which might be useful), when conducting unit tests for metrics, we can use this package: https://github.com/open-telemetry/opentelemetry-go/tree/main/sdk/metric/metricdata/metricdatatest

In fact, I haven't started the review of unit tests yet. (Reason: The functional part has not yet been fully reviewed).

@minimAluminiumalism
Copy link
Copy Markdown
Author

Since this PR includes 2 components, I suggest implementing one of them first, as it will be easier to get through during both implementation and review.

I've removed the implementation of otlpmetrichttp, please review it.

@flc1125 flc1125 changed the title feat: add self-observability metrics to otlpmetricgrpc and otlpmetrichttp metric exporters feat: add self-observability metrics to otlpmetrichttp metric exporters Aug 7, 2025
@flc1125 flc1125 changed the title feat: add self-observability metrics to otlpmetrichttp metric exporters feat: add self-observability metrics to otlpmetricgrpc metric exporters Aug 7, 2025
Comment thread exporters/otlp/otlpmetric/otlpmetrichttp/exporter.go Outdated
Copy link
Copy Markdown
Member

@flc1125 flc1125 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution.

@minimAluminiumalism
Copy link
Copy Markdown
Author

@pellared @flc1125 I've fixed the CI pipeline falilures caused by docs, please re-run it.

@minimAluminiumalism
Copy link
Copy Markdown
Author

@flc1125 Conflicts resolved, re-run the pipeline please :)

@minimAluminiumalism
Copy link
Copy Markdown
Author

@flc1125 Maybe it's time to merge this PR? I'll implement it of otlpmetrichttp next.

@flc1125
Copy link
Copy Markdown
Member

flc1125 commented Aug 25, 2025

@flc1125 Maybe it's time to merge this PR? I'll implement it of otlpmetrichttp next.

Sorry, I was on vacation last week, and there have been some significant recent changes that I need to review.

@flc1125
Copy link
Copy Markdown
Member

flc1125 commented Aug 25, 2025

Please use this component https://github.com/open-telemetry/opentelemetry-go/tree/main/exporters/stdout/stdouttrace as a reference; there have been some recent optimizations, and we need to adjust accordingly.

You can view the change log via this link: https://github.com/open-telemetry/opentelemetry-go/commits/main/exporters/stdout/stdouttrace

Please refer to the adjustment for this range: 2ab7d4d...c2a6172

@pellared
Copy link
Copy Markdown
Member

pellared commented Sep 1, 2025

@minimAluminiumalism PTAL #7272

@minimAluminiumalism
Copy link
Copy Markdown
Author

@pellared @flc1125 Apologies for the delay—I’ve been blocked by other priorities and will aim to complete this PR by next week.

@flc1125
Copy link
Copy Markdown
Member

flc1125 commented Sep 11, 2025

Hi, since this process involves specification adjustments and historical review records (which may contain invalid review suggestions), it’s impossible to tell which items need attention amid the large volume of information.

Could we create a new PR based on the current branch before preparing for the review, so that subsequent reviews can proceed more smoothly?

Thanks~

@MrAlias MrAlias added this to the v1.39.0 milestone Sep 15, 2025
@MrAlias
Copy link
Copy Markdown
Contributor

MrAlias commented Sep 15, 2025

@minimAluminiumalism please take a look at #7012. I have updated the issue with a check list of items from our project Observability guidelines that need to be completed in this PR.

As @flc1125 pointed out, you may find it easier to open a new PR. But feel free to update this PR as well. If you plan to update this PR, please mark it as a draft while you are working to address the items listed in #7012 and the Observability guidelines.

@minimAluminiumalism
Copy link
Copy Markdown
Author

@minimAluminiumalism please take a look at #7012. I have updated the issue with a check list of items from our project Observability guidelines that need to be completed in this PR.

As @flc1125 pointed out, you may find it easier to open a new PR. But feel free to update this PR as well. If you plan to update this PR, please mark it as a draft while you are working to address the items listed in #7012 and the Observability guidelines.

Ok, I'll submit a new PR.

@MrAlias
Copy link
Copy Markdown
Contributor

MrAlias commented Oct 7, 2025

@minimAluminiumalism just wanted to check in on this. Are you still able to update this PR or open another PR to address the instrumentation requirements?

@minimAluminiumalism
Copy link
Copy Markdown
Author

@minimAluminiumalism just wanted to check in on this. Are you still able to update this PR or open another PR to address the instrumentation requirements?

I just finished my vacation. And I'll finish this PR and submit it in the next few days.

@pellared
Copy link
Copy Markdown
Member

@minimAluminiumalism, any update on your side? Are you able to update the PR?

@MrAlias MrAlias modified the milestones: v1.39.0, v1.40.0 Dec 4, 2025
@MrAlias MrAlias modified the milestones: v1.40.0, v1.41.0 Jan 29, 2026
@MrAlias MrAlias modified the milestones: v1.41.0, v1.42.0 Feb 26, 2026
@pellared
Copy link
Copy Markdown
Member

pellared commented Mar 4, 2026

@minimAluminiumalism, kindly reminder.

@pellared pellared removed this from the v1.42.0 milestone Mar 4, 2026
dataPointCount := countDataPoints(rm)
startTime := time.Now()

em.inflight.Add(ctx, dataPointCount, em.attrs...)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


duration := time.Since(startTime).Seconds()

attrs := make([]attribute.KeyValue, len(em.attrs), len(em.attrs)+1)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should pool attribute slices here.

"go.opentelemetry.io/otel/metric"
"go.opentelemetry.io/otel/sdk"
"go.opentelemetry.io/otel/sdk/metric/metricdata"
semconv "go.opentelemetry.io/otel/semconv/v1.36.0"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to semconv 1.40

dataPointCount := countDataPoints(rm)
startTime := time.Now()

em.inflight.Add(ctx, dataPointCount, em.attrs...)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should check if each instrument is Enabled(ctx) prior to making Add or Record calls.

@dashpole
Copy link
Copy Markdown
Contributor

I'll pick this PR up to get it over the finish line since it hasn't been updated in a few months. Thanks for all your hard work @minimAluminiumalism!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metrics SDK observability - otlpmetricgrpc exporter metrics

5 participants