feat: logs SDK observability - otlploggrpc exporter metrics by yumosx · Pull Request #7084 · open-telemetry/opentelemetry-go

yumosx · 2025-07-25T10:05:32Z

codecov · 2025-07-25T10:15:41Z

Codecov Report

❌ Patch coverage is 96.15385% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.0%. Comparing base (e26cebf) to head (1569d2b).
⚠️ Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...log/otlploggrpc/internal/observ/instrumentation.go	94.5%	6 Missing and 2 partials ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #7084     +/-   ##
=======================================
+ Coverage   82.9%   83.0%   +0.1%     
=======================================
  Files        267     270      +3     
  Lines      24997   25201    +204     
=======================================
+ Hits       20743   20942    +199     
- Misses      3878    3881      +3     
- Partials     376     378      +2

Files with missing lines	Coverage Δ
exporters/otlp/otlplog/otlploggrpc/client.go	`93.8% <100.0%> (+1.9%)`	⬆️
...lp/otlplog/otlploggrpc/internal/counter/counter.go	`100.0% <100.0%> (ø)`
exporters/otlp/otlplog/otlploggrpc/internal/x/x.go	`100.0% <100.0%> (ø)`
...log/otlploggrpc/internal/observ/instrumentation.go	`94.5% <94.5%> (ø)`

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

yumosx · 2025-07-25T11:56:29Z

Good suggestion, done.

- use atomic.Int64 - use ErrorType

flc1125 · 2025-07-28T12:51:52Z

Could you add to the CHANGELOG

yumosx · 2025-07-28T12:56:26Z

Could you add to the CHANGELOG

sure.

yumosx · 2025-07-28T16:25:57Z

this ci/codecov seems to need to be run again.

Warning: Failed to download action 'https://api.github.com/repos/actions/download-artifact/tarball/d3f86a106a0bac45b974a628896c90dbdf5c8093'. Error: Response status code does not indicate success: 429 (Too Many Requests). 3400:2A265C:4E71A:632CD:6887A211
Warning: Back off 21.806 seconds before retry.
Warning: Failed to download action 'https://api.github.com/repos/actions/download-artifact/tarball/d3f86a106a0bac45b974a628896c90dbdf5c8093'. Error: Response status code does not indicate success: 429 (Too Many Requests). 3400:2A265C:4E978:63665:6887A227
Warning: Back off 12.793 seconds before retry.
Error: Response status code does not indicate success: 429 (Too Many Requests).

flc1125 · 2025-07-29T00:42:23Z

this ci/codecov seems to need to be run again.

I have already retried.

# Conflicts: # exporters/otlp/otlplog/otlploggrpc/client_test.go

flc1125 · 2025-08-29T14:11:52Z

This PR is expected to wait until after version 1.38; we need to wait to avoid blocking the 1.38 release.

yumosx · 2025-08-29T14:34:59Z

This PR is expected to wait until after version 1.38; we need to wait to avoid blocking the 1.38 release.

OK

MrAlias

This is a high-quality PR. Thank you for contributing it and thank you for following the contribution guidelines.

There are some minor naming issues that need to be addressed. But there are two larger issues that still need attention:

There are no benchmark tests showing the performance of the Instrumentation methods
The performance of adding the RPC status code attribute needs to be better evaluated

MrAlias · 2025-09-10T21:58:02Z

+//
+// The number of successful exports is provided as success. Any error that is encountered is provided as error
+// The code of last gRPC requests performed in scope of this export call.
+type ExportSpanDone func(err error, success int64, code codes.Code)


Suggested change

type ExportSpanDone func(err error, success int64, code codes.Code)

type ExportLogsDone func(err error, success int64, code codes.Code)

Docs will need an update as well.

MrAlias · 2025-09-10T21:58:48Z

+
+// ExportSpans instruments the ExportSpans method of the exporter. It returns a
+// function that needs to be deferred so it is called when the method returns.
+func (i *Instrumentation) ExportSpans(ctx context.Context, count int64) ExportSpanDone {


Suggested change

func (i *Instrumentation) ExportSpans(ctx context.Context, count int64) ExportSpanDone {

func (i *Instrumentation) ExportLogs(ctx context.Context, count int64) ExportLogsDone {

Docs will need an update as well.

MrAlias · 2025-09-10T21:59:18Z

+
+	logExportedMetric, e := otelconv.NewSDKExporterLogExported(m)
+	if e != nil {
+		e = fmt.Errorf("failed to create span exported metric: %w", e)


Suggested change

e = fmt.Errorf("failed to create span exported metric: %w", e)

e = fmt.Errorf("failed to create log exported metric: %w", e)

MrAlias · 2025-09-10T21:59:37Z

+
+	logInflightMetric, e := otelconv.NewSDKExporterLogInflight(m)
+	if e != nil {
+		e = fmt.Errorf("failed to create span inflight metric: %w", e)


Suggested change

e = fmt.Errorf("failed to create span inflight metric: %w", e)

e = fmt.Errorf("failed to create log inflight metric: %w", e)

MrAlias · 2025-09-10T22:03:34Z

+// NewInstrumentation returns instrumentation for otlplog grpc exporter.
+func NewInstrumentation(
+	name, componentName string,
+	componentType otelconv.ComponentTypeAttr,


This will always be otelconv.ComponentTypeOtlpGRPCLogExporter right? Do we need to accept it as a parameter?

Yes, I will adjust it later.

flc1125 · 2025-09-11T01:18:30Z

Because this process involves some specification adjustments and historical reviews (which may even include invalid review suggestions), the information of other approvers can no longer be viewed during the review, causing some inconvenience. Could we create a new PR based on the current branch to reduce distracting information.

yumosx · 2025-09-11T01:30:55Z

Sure. I will open a new PR based on it later.

flc1125 · 2025-09-11T01:10:38Z

+//
+// This package is used to generate unique IDs while allowing testing packages
+// to reset the counter.
+package counter // import "go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/counter"


Since we already have this template, we can consider using it for generation to ensure unified maintenance. https://github.com/open-telemetry/opentelemetry-go/tree/main/internal/shared/counter

flc1125 · 2025-09-11T01:29:21Z


 - Add `WithInstrumentationAttributeSet` option to `go.opentelemetry.io/otel/log`, `go.opentelemetry.io/otel/metric`, and `go.opentelemetry.io/otel/trace` packages.
  This provides a concurrent-safe and performant alternative to `WithInstrumentationAttributes` by accepting a pre-constructed `attribute.Set`. (#7287)
+- - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`.


Suggested change

- - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`.

- Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`.

flc1125 · 2025-09-11T01:36:56Z

+	logInflightMetric, e := otelconv.NewSDKExporterLogInflight(m)
+	if e != nil {
+		e = fmt.Errorf("failed to create span inflight metric: %w", e)
+		otel.Handle(e)


Since we’re returning the err, I’m thinking we can skip logging this part and let the outer caller decide uniformly.

The same applies to the others.

good suggestion, this error should be handled by the caller.

yumosx · 2025-09-11T16:42:44Z

@open-telemetry/go-approvers , all done, I create a new PR based on the current branch , place check.

Thank you all for your suggestions on this PR. ❤️

flc1125 · 2025-09-12T00:46:41Z

I create a new PR based on the current branch , place check.

Since a new one has been created, we can close the current PR.

yumosx · 2025-09-12T00:59:34Z

done.

This PR adds support for experimental metrics in `otlploggrpc` - `otel.sdk.exporter.log.inflight` - `otel.sdk.exporter.log.exported` - `otel.sdk.exporter.operation.duration` References: - #7084 - #7019 - [Follow guidelines](https://github.com/open-telemetry/opentelemetry-go/blob/a5dcd68ebb2f3669f7685ac7b0f3f1624251a381/CONTRIBUTING.md#encapsulation). ----- ```txt goos: darwin goarch: arm64 pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc cpu: Apple M3 │ disabled.txt │ enabled.txt │ │ sec/op │ sec/op vs base │ ExporterExportLogs/Observability-8 681.5µ ± 3% 684.3µ ± 6% ~ (p=0.315 n=10) │ disabled.txt │ enabled.txt │ │ B/op │ B/op vs base │ ExporterExportLogs/Observability-8 672.8Ki ± 0% 673.6Ki ± 1% ~ (p=0.247 n=10) │ disabled.txt │ enabled.txt │ │ allocs/op │ allocs/op vs base │ ExporterExportLogs/Observability-8 9.224k ± 0% 9.232k ± 0% +0.09% (p=0.000 n=10) ``` ----- ```txt goos: darwin goarch: arm64 pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/observ cpu: Apple M3 │ bench.txt │ │ sec/op │ InstrumentationExportLogs/NoError-8 162.6n ± 3% InstrumentationExportLogs/PartialError-8 705.5n ± 5% InstrumentationExportLogs/FullError-8 592.1n ± 1% geomean 408.0n │ bench.txt │ │ B/op │ InstrumentationExportLogs/NoError-8 152.0 ± 0% InstrumentationExportLogs/PartialError-8 697.0 ± 0% InstrumentationExportLogs/FullError-8 616.0 ± 0% geomean 402.6 │ bench.txt │ │ allocs/op │ InstrumentationExportLogs/NoError-8 3.000 ± 0% InstrumentationExportLogs/PartialError-8 10.00 ± 0% InstrumentationExportLogs/FullError-8 8.000 ± 0% geomean 6.214 ``` ----- ```txt pkg: go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc/internal/observ cpu: Apple M3 │ parse_target.txt │ │ sec/op │ ParseTarget/HostName-8 38.00n ± ∞ ¹ ParseTarget/HostPort-8 51.33n ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 44.74n ± ∞ ¹ ParseTarget/IPv4WithPort-8 62.56n ± ∞ ¹ ParseTarget/IPv6Bare-8 94.89n ± ∞ ¹ ParseTarget/IPv6Bracket-8 93.78n ± ∞ ¹ ParseTarget/IPv6WithPort-8 57.57n ± ∞ ¹ ParseTarget/UnixSocket-8 8.329n ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 9.082n ± ∞ ¹ ParseTarget/Passthrough-8 58.06n ± ∞ ¹ geomean 40.64n ¹ need >= 6 samples for confidence interval at level 0.95 │ parse_target.txt │ │ B/op │ ParseTarget/HostName-8 48.00 ± ∞ ¹ ParseTarget/HostPort-8 48.00 ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 16.00 ± ∞ ¹ ParseTarget/IPv4WithPort-8 48.00 ± ∞ ¹ ParseTarget/IPv6Bare-8 16.00 ± ∞ ¹ ParseTarget/IPv6Bracket-8 16.00 ± ∞ ¹ ParseTarget/IPv6WithPort-8 48.00 ± ∞ ¹ ParseTarget/UnixSocket-8 0.000 ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 0.000 ± ∞ ¹ ParseTarget/Passthrough-8 48.00 ± ∞ ¹ geomean ² ¹ need >= 6 samples for confidence interval at level 0.95 ² summaries must be >0 to compute geomean │ parse_target.txt │ │ allocs/op │ ParseTarget/HostName-8 1.000 ± ∞ ¹ ParseTarget/HostPort-8 1.000 ± ∞ ¹ ParseTarget/IPv4WithoutPort-8 1.000 ± ∞ ¹ ParseTarget/IPv4WithPort-8 1.000 ± ∞ ¹ ParseTarget/IPv6Bare-8 1.000 ± ∞ ¹ ParseTarget/IPv6Bracket-8 1.000 ± ∞ ¹ ParseTarget/IPv6WithPort-8 1.000 ± ∞ ¹ ParseTarget/UnixSocket-8 0.000 ± ∞ ¹ ParseTarget/UnixAbstractSocket-8 0.000 ± ∞ ¹ ParseTarget/Passthrough-8 1.000 ± ∞ ¹ geomean ² ¹ need >= 6 samples for confidence interval at level 0.95 ² summaries must be >0 to compute geomean ``` --------- Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>

yumosx added 3 commits July 24, 2025 11:22

add the otlplogrpc selfObservaility

ed73a66

merge upstream

4aa0a5b

Add self-observability support for otlplog grpc exporter

87f64f3

yumosx requested review from MrAlias, XSAM, dashpole, dmathieu, flc1125 and pellared as code owners July 25, 2025 10:05

yumosx added 2 commits July 25, 2025 18:06

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

bd9de33

commit

56be45b

flc1125 requested changes Jul 25, 2025

View reviewed changes

Comment thread exporters/otlp/otlplog/otlploggrpc/internal/x/x.go

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

flc1125 reviewed Jul 25, 2025

View reviewed changes

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

add x_test.go and update selfObsever

2961556

dashpole reviewed Jul 25, 2025

View reviewed changes

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

yumosx added 3 commits July 26, 2025 01:06

- compute post on initialization

97a7d77

- use atomic.Int64 - use ErrorType

fix the data race

030120c

updated

0c7a55b

yumosx added 5 commits July 28, 2025 21:14

add changlog entry and readme.md

be6242e

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

7c31573

fix spell

2e1da4a

add the error tests

d228db8

improve error handling

7a41831

flc1125 reviewed Jul 29, 2025

View reviewed changes

Comment thread exporters/otlp/otlplog/otlploggrpc/client.go Outdated

yumosx added 3 commits August 21, 2025 01:23

refactor: avoid assigning presetAttrs when metric initialization fails

d9e8dff

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

4bbfed0

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

0b0f411

flc1125 reviewed Aug 27, 2025

View reviewed changes

yumosx added 3 commits August 28, 2025 21:46

perf: use addSet instead of add for better performance

a32dd6e

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

1736a9e

# Conflicts: # exporters/otlp/otlplog/otlploggrpc/client_test.go

fix the missing tests

b778294

yumosx force-pushed the yumosx-self-mlog branch from b5288ca to b778294 Compare August 28, 2025 14:38

mahendrabishnoi2 mentioned this pull request Aug 28, 2025

REQUEST: New membership for mahendrabishnoi2 open-telemetry/community#2955

Closed

6 tasks

yumosx requested a review from flc1125 August 29, 2025 09:21

yumosx added 6 commits September 9, 2025 23:30

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

947bf9c

update go mod

ef06e5d

refactor(self-observability): align code with style guide

eb8493a

fix the changlog check

16eaf3c

refactor: rename the selfoberv to observ and direct use the inst

74033bc

Merge remote-tracking branch 'upstream/main' into yumosx-self-mlog

1569d2b

MrAlias reviewed Sep 10, 2025

View reviewed changes

flc1125 reviewed Sep 11, 2025

View reviewed changes

yumosx mentioned this pull request Sep 11, 2025

feat: logs SDK observability - otlploggrpc exporter metrics #7353

Merged

yumosx closed this Sep 12, 2025

yumosx mentioned this pull request Sep 12, 2025

[exporter/otlp] Attempts to parse port from unix socket endpoint and fails open-telemetry/opentelemetry-collector#13826

Closed

	type ExportSpanDone func(err error, success int64, code codes.Code)
	type ExportLogsDone func(err error, success int64, code codes.Code)

	func (i *Instrumentation) ExportSpans(ctx context.Context, count int64) ExportSpanDone {
	func (i *Instrumentation) ExportLogs(ctx context.Context, count int64) ExportLogsDone {

	e = fmt.Errorf("failed to create span exported metric: %w", e)
	e = fmt.Errorf("failed to create log exported metric: %w", e)

	e = fmt.Errorf("failed to create span inflight metric: %w", e)
	e = fmt.Errorf("failed to create log inflight metric: %w", e)

	- - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`.
	- Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`.

Conversation

yumosx commented Jul 25, 2025

Uh oh!

codecov bot commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yumosx commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flc1125 commented Jul 28, 2025

Uh oh!

yumosx commented Jul 28, 2025

Uh oh!

yumosx commented Jul 28, 2025

Uh oh!

flc1125 commented Jul 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

flc1125 commented Aug 29, 2025

Uh oh!

yumosx commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MrAlias left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flc1125 commented Sep 11, 2025

Uh oh!

yumosx commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yumosx commented Sep 11, 2025

Uh oh!

flc1125 commented Sep 12, 2025

Uh oh!

yumosx commented Sep 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

codecov bot commented Jul 25, 2025 •

edited

Loading

yumosx commented Aug 29, 2025 •

edited

Loading

MrAlias left a comment •

edited

Loading

yumosx commented Sep 11, 2025 •

edited

Loading