tools/coredump: allow running tests on darwin by korniltsev · Pull Request #621 · open-telemetry/opentelemetry-ebpf-profiler

korniltsev · 2025-07-16T09:37:45Z

~~Similar to #612~~

~~This will eventually help allowing running some unit tests (for example coredump tests) without spawning a full linux OS on non linux systems (at least darwin, ~~maybe windows~~)~~

This PR makes running coredump tests on darwin possible

GODEBUG=asyncpreemptoff=1 go test -v ./tools/coredump/.

florianl · 2025-07-16T09:48:46Z

I think, there are more changes needed to reach this point:

$ GOOS=windows go build
# go.opentelemetry.io/ebpf-profiler/libpf/pfelf/internal/mmap
libpf/pfelf/internal/mmap/mmap.go:49:17: undefined: syscall.Munmap
libpf/pfelf/internal/mmap/mmap.go:116:23: undefined: syscall.Mmap
libpf/pfelf/internal/mmap/mmap.go:116:63: undefined: syscall.PROT_READ
libpf/pfelf/internal/mmap/mmap.go:116:82: undefined: syscall.MAP_SHARED
# go.opentelemetry.io/ebpf-profiler/rlimit
rlimit/rlimit.go:17:20: undefined: unix.Rlimit
rlimit/rlimit.go:18:19: undefined: unix.Rlimit
rlimit/rlimit.go:19:13: undefined: unix.RLIM_INFINITY
rlimit/rlimit.go:20:13: undefined: unix.RLIM_INFINITY
rlimit/rlimit.go:23:17: undefined: unix.Prlimit
rlimit/rlimit.go:23:33: undefined: unix.RLIMIT_MEMLOCK
rlimit/rlimit.go:28:18: undefined: unix.Setrlimit
rlimit/rlimit.go:28:33: undefined: unix.RLIMIT_MEMLOCK

rockdaboot · 2025-07-16T09:55:24Z

I think, there are more changes needed to reach this point:

Maybe change the title and say non-linux open source systems!? I guess supporting Windows builds just adds more complexity without gaining any value.

florianl · 2025-07-16T09:58:24Z

It is not just Windows.

$ GOOS=darwin go build
# go.opentelemetry.io/ebpf-profiler/rlimit
rlimit/rlimit.go:23:17: undefined: unix.Prlimit
$ GOOS=netbsd go build
# go.opentelemetry.io/ebpf-profiler/rlimit
rlimit/rlimit.go:23:17: undefined: unix.Prlimi

korniltsev · 2025-07-16T10:01:32Z

I think, there are more changes needed to reach this point:

yes, they are in different packages. rlimit and processmanager/ebpf are my next targets.

I want the change to be small and self contained and revertable just in case

rockdaboot · 2025-07-16T10:02:23Z

@korniltsev Please be more specific in the PR title.

korniltsev · 2025-07-16T10:17:09Z

@korniltsev Please be more specific in the PR title.

Thanks, updated

korniltsev · 2025-07-16T10:18:43Z

I guess supporting Windows builds just adds more complexity without gaining any value.

It would gain the same value as darwin.
I have not looked into windows yet, darwin will probably be slightly easier as it does support mmap.

florianl · 2025-07-16T11:42:14Z

maybe rename the file to debug_linux.go

korniltsev · 2025-07-16T13:30:15Z

Just had an idea similar tohttps://github.com//pull/625. Maybe instead of introducing build tags, we can split the package - move the ptrace specific code into a separate ptrace package

christos68k · 2025-07-16T14:03:34Z

Before approving this, I would like to have the complete picture in terms of how the code structure is changing and also the build system. In principle, I'm not against enabling limited compilation on non-Linux systems, but not if it comes with a lot of extra complexity and potential support headaches.

The profiler only works on Linux [1] and that's where the focus should be. @korniltsev If you have multiple PRs in mind, then I'd delay approving/merging until you open all of them so we can look at them as a whole.

[1] Technically it also works inside Docker-on-macOS as that spins up a Linux kernel that supports eBPF that's shared across containers running on the same macOS host. Maybe it also works on WSL2.

korniltsev · 2025-07-16T14:29:06Z

I dont have any more new PRs in mind regarding this.

There are 2 things I want to do.

There is a rlimit: allow compiling rlimit on non-linux systems #624 with an alternative proposal WIP: split processmanager/ebpf in two packages: interface and implementation #625
tools/coredump: allow running tests on darwin #621

Together they allow compiling and running coredump tests.

The tests fail with an unexpected SIGURG signal - that would require another fix.

Thats it. This will enable running a big and useful subset of unit tests without spawning virtual machines.

korniltsev · 2025-07-16T14:55:45Z

To add a bit more on my previous comment:

I see 2 possible solutions to what I am trying to achieve.

guard linux specific code with go build tags
Split packages both processmanager/ebpf and process in half - keep the base interfaces inplace and move the specific implementations to a separate package, for example process/ptrace and processmanager/ebpf/cilium. I kind of like this idea more now. It's should be a good change on it's own - separating concerns.

Do you have any other ideas how this could be done?

…nonlinux

korniltsev · 2025-07-25T10:39:34Z

lets maybe try a different approach - I've closed all my other PRs and consolidated everything here.

Now the tests can be run with GODEBUG=asyncpreemptoff=1 go test -v ./tools/coredump/.

I'd appreciate another round of reviews and discussions. 🙏

…nonlinux

fabled · 2025-07-28T15:24:59Z

+
+import "debug/elf"
+
+const currentMachine = elf.EM_X86_64


I would probably move these currentMachine things to pfelf as CurrentMachine. They are occasionally needed in other parts too.

Applied. Though I could not find where to use them in other parts after a quick search.

fabled · 2025-07-28T15:27:32Z

+        uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
+        with:
+          path: tools/coredump/modulecache
+          key: coredumps-macos-${{ hashFiles('tools/coredump/testdata/*/*.json') }}


Does the key need to be separate from others? I think this could be shared, because only the actual downloaded files are in the cache. There's nothing linux/mac specific.

Thanks. Replaced the key with arm64 to be shared.

I don't fully understand the restore-keys options though. It feels like they are never used

fabled · 2025-07-28T15:30:32Z

-	updatePoolQueueCap = 8
 )

 // EbpfHandler provides the functionality to interact with eBPF maps.


If I understand correctly, this and rest of this PR is about separating EbpfHandler interface out from the processmanager/ebpf so that the interface can be included elsewhere without pulling in the actual ebpf code which does not compile?

Could we do this the otherway around? Just move this interface to some other package like processmanager/ebpfapi? Need to also check how this kind of split is done elsewhere.

If I understand correctly, this and rest of this PR is about separating EbpfHandler interface out from the processmanager/ebpf so that the interface can be included elsewhere without pulling in the actual ebpf code which does not compile?

Yes, this is correct in regards the changes in the pmebpf package.
There are some other changes in the process package, but they are slightly different - solving the compilation issues with build tags. I do think it makes sense to split process into processapi in a similar way you suggested ebpfapi - let me know if you think we should do this within the same change or as a separate or at all.

Could we do this the otherway around? Just move this interface to some other package like processmanager/ebpfapi? Need to also check how this kind of split is done elsewhere.

Yes, this should work. Applied

fabled

LGTM

process: allow compiling on nonlinux systems

499551b

korniltsev marked this pull request as ready for review July 16, 2025 09:41

korniltsev requested review from a team as code owners July 16, 2025 09:41

rockdaboot approved these changes Jul 16, 2025

View reviewed changes

korniltsev changed the title ~~process: allow compiling on nonlinux systems~~ process: allow compiling on darwin Jul 16, 2025

florianl reviewed Jul 16, 2025

View reviewed changes

rename debug.go to debug_linuxx.go

d1c72d9

korniltsev added 6 commits July 25, 2025 16:20

Merge remote-tracking branch 'upstream/main' into korniltsev/process_…

50289fe

…nonlinux

split processmanager/ebpf

66ffc02

revert some changes

f215f8d

lint

d67372a

revert some changes

9b9d135

revert some changes

30e6ea9

korniltsev changed the title ~~process: allow compiling on darwin~~ tools/coredump: allow running on darwin Jul 25, 2025

add macos job

a34798f

korniltsev requested review from florianl and rockdaboot July 25, 2025 10:39

fmt

47b3473

korniltsev changed the title ~~tools/coredump: allow running on darwin~~ tools/coredump: allow running tests on darwin Jul 25, 2025

korniltsev added 2 commits July 25, 2025 21:27

Merge remote-tracking branch 'upstream/main' into korniltsev/process_…

4ed4604

…nonlinux

Merge remote-tracking branch 'upstream/main' into korniltsev/process_…

ec1fe68

…nonlinux

fabled reviewed Jul 28, 2025

View reviewed changes

korniltsev added 3 commits July 29, 2025 10:20

ebpfapi - the other way around

f500d40

move machine

72c0c75

update gh actions cache key for macos job

c5568a7

fabled approved these changes Aug 6, 2025

View reviewed changes

christos68k approved these changes Aug 6, 2025

View reviewed changes

florianl approved these changes Aug 6, 2025

View reviewed changes

christos68k merged commit 66e4959 into open-telemetry:main Aug 6, 2025
28 checks passed

This was referenced Aug 14, 2025

fix golabels tests parca-dev/opentelemetry-ebpf-profiler#92

Closed

fix golabels test parca-dev/opentelemetry-ebpf-profiler#97

Closed

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025

tools/coredump: allow running tests on darwin (open-telemetry#621)

9730335

gnurizen mentioned this pull request Sep 18, 2025

update4 parca-dev/opentelemetry-ebpf-profiler#138

Merged

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025

tools/coredump: allow running tests on darwin (open-telemetry#621)

b910521

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025

tools/coredump: allow running tests on darwin (open-telemetry#621)

cac6430

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025

tools/coredump: allow running tests on darwin (open-telemetry#621)

ff303d4

gnurizen mentioned this pull request Mar 9, 2026

Optimize distro QEMU tests to be more efficient parca-dev/opentelemetry-ebpf-profiler#228

Closed

Conversation

korniltsev commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

florianl commented Jul 16, 2025

Uh oh!

rockdaboot commented Jul 16, 2025

Uh oh!

florianl commented Jul 16, 2025

Uh oh!

korniltsev commented Jul 16, 2025

Uh oh!

rockdaboot commented Jul 16, 2025

Uh oh!

korniltsev commented Jul 16, 2025

Uh oh!

korniltsev commented Jul 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

korniltsev commented Jul 16, 2025

Uh oh!

christos68k commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

korniltsev commented Jul 16, 2025

Uh oh!

korniltsev commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

korniltsev commented Jul 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fabled left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

korniltsev commented Jul 16, 2025 •

edited

Loading

christos68k commented Jul 16, 2025 •

edited

Loading

korniltsev commented Jul 16, 2025 •

edited

Loading