Skip to content

golabels: ignore attach when Go plugins are loaded#1314

Merged
christos68k merged 3 commits into
open-telemetry:mainfrom
parca-dev:fix/golabels-plugin-double-detach
Apr 3, 2026
Merged

golabels: ignore attach when Go plugins are loaded#1314
christos68k merged 3 commits into
open-telemetry:mainfrom
parca-dev:fix/golabels-plugin-double-detach

Conversation

@gnurizen
Copy link
Copy Markdown
Contributor

@gnurizen gnurizen commented Apr 1, 2026

When a Go process loads shared libraries via plugin.Open(), the profiler creates separate golabels interpreter instances for each Go ELF file in the process. Since Go plugins must be built with the same Go version as the host binary and share the runtime, they produce identical GoLabelsOffsets — but both instances write and delete the same PID-keyed eBPF map entry.

The second Detach call fails with "key does not exist" because the first instance already removed it.

This fixes the issue by ignoring the attach of Go plugins, since the proc data would be the same there's no point in maintaining a golabels instance for it.

Fixes #1301

@gnurizen gnurizen requested review from a team as code owners April 1, 2026 19:12
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Apr 1, 2026

CLA Signed

The committers listed above are authorized under a signed CLA.

@gnurizen gnurizen marked this pull request as draft April 1, 2026 19:37
@gnurizen gnurizen force-pushed the fix/golabels-plugin-double-detach branch 2 times, most recently from ef89187 to 760c733 Compare April 1, 2026 19:39
When a Go process loads shared libraries via plugin.Open(), the profiler
creates separate golabels interpreter instances for each Go ELF file in
the process. Since they all share the same PID-keyed eBPF map entry, the
second Detach fails with "key does not exist".

Fix by wrapping the error in DeleteProcData with %w so callers can
inspect it, and ignoring ErrKeyNotExist in golabels Detach.

Fixes open-telemetry#1301
@gnurizen gnurizen marked this pull request as ready for review April 1, 2026 19:47
Comment thread interpreter/golabels/golabels.go Outdated
Copy link
Copy Markdown
Contributor

@rogercoll rogercoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Maybe add a debug log line if something actually goes wrong with golabels in a multi-plugin process?

Comment thread interpreter/golabels/golabels.go Outdated
Comment thread interpreter/golabels/golabels.go Outdated
Comment on lines +49 to +52
if err := ebpf.DeleteProcData(libpf.GoLabels, pid); err != nil && !errors.Is(err, cebpf.ErrKeyNotExist) {
return err
}
return nil
Copy link
Copy Markdown
Member

@florianl florianl Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would also incorporate #1314 (comment):

Suggested change
if err := ebpf.DeleteProcData(libpf.GoLabels, pid); err != nil && !errors.Is(err, cebpf.ErrKeyNotExist) {
return err
}
return nil
err := ebpf.DeleteProcData(libpf.GoLabels, pid)
switch {
case err == nil: return
case errors.Is(err, cebpf.ErrKeyNotExist):
log.Debugf("Entry for PID %d already removed", pid)
return nil
default:
return err
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need the err == nil case I think, I pushed a commit to add logging.

Copy link
Copy Markdown
Member

@christos68k christos68k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread interpreter/golabels/golabels.go Outdated

func (d *data) Detach(ebpf interpreter.EbpfHandler, pid libpf.PID) error {
return ebpf.DeleteProcData(libpf.GoLabels, pid)
// Go plugins share the runtime with the main binary, so multiple Go ELF
Copy link
Copy Markdown
Member

@christos68k christos68k Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It it worth further clarifying that Go does not support unloading plugins here? If one doesn't know that then one could think that we still have an issue.

Go plugins can't change go labels version/offset information nor the
TLS information so ignore them by detecting when a go exe is a shared
object.
@gnurizen
Copy link
Copy Markdown
Contributor Author

gnurizen commented Apr 3, 2026

I completely flipped this on its head and think its better to just ignore Go plugins, please take another look, thanks!

@gnurizen gnurizen changed the title golabels: tolerate duplicate detach when Go plugins are loaded golabels: ignore attach when Go plugins are loaded Apr 3, 2026
Copy link
Copy Markdown
Member

@christos68k christos68k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@christos68k christos68k merged commit 582a2cf into open-telemetry:main Apr 3, 2026
32 checks passed
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request May 4, 2026
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request May 5, 2026
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request May 12, 2026
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

golabels: Failed to unload interpreter when using plugin/shared libraries

4 participants