Refactor symbol caching by fabled · Pull Request #635 · open-telemetry/opentelemetry-ebpf-profiler

fabled · 2025-07-20T16:41:28Z

This creates libpf.Frame with the libpf.Trace frame data and the relevant metadata:

to not use excessive memory the libpf.Frame is interned using the unique package in most places
refactors the data structures and APIs so that proper symbolization data is always available for reporter module
remove the now unneeded LRU from reporter
removes the internal dependency of synthesized FileID/LineNo (if some reporter needs this always, they should synthesize them when needed)
fixes Restructure of symbols information caching #384

Follow up steps:

refactor the Trace caching: cache host.Frame to libpf.Frames in a LRU, potentially remove the host.Trace to libpf.Trace LRU completing
make libpf.Frame have mappings/executable info as a separate interned data field to address Executable metadata missing for some executables on hosts with a large number of executables #250

This joins the Frame metadata to libpf.Frame: - removes the internal dependency of synthesized FileID/LineNo (if some reporter needs to synthesize these still, they need to now do it them selves based on the trace/symbol data) - removes LRU from the base reporter and allows the profiler core to always provide symbol information To not use excessive memory the libpf.Frame is interned using the unique package. TODO: - add caching for ebpf frame -> libpf.Frame and check if the tracehandler trace LRU can be removed

I removed this as I was planning to include the tracehandler cache changes making it redundant, but as it is wort a separate PR lets add this back for now.

christos68k

Did a pass, looks good to me. I will do a final pass tomorrow.

florianl

I like the simplicity and reduction of API functions this work introduces. Thanks for working on this!

florianl · 2025-07-29T14:26:25Z

-		SourceLine:   libpf.SourceLineno(lineNo),
+	frames.Append(&libpf.Frame{
+		Type: libpf.GoFrame,
+		//TODO: File: convert the frame.File (host.FileID) to libpf.FileID here


As Go frames are treated like any other interpreted frame, I think we can skip File here. WDYT?

Suggested change

//TODO: File: convert the frame.File (host.FileID) to libpf.FileID here

I think it makes sense to provide it here as a follow up as it provides additional information and this is a file backed mapping.

open-telemetry/opentelemetry-ebpf-profiler#635

Update go.opentelemetry.io/ebpf-profiler to use a tagged version instead of a commit. As open-telemetry/opentelemetry-ebpf-profiler#635 removed libpf.FrameID, also move its functionality to this package as it is essential for its functionality. Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

open-telemetry/opentelemetry-ebpf-profiler#635

* backport: replace per-fileID LRU with a global LRU open-telemetry/opentelemetry-ebpf-profiler#529 * backport: interpreters, reporter: intern symbolization strings open-telemetry/opentelemetry-ebpf-profiler#563 * Disable Go interpreter because we are doing Go symbolization remotely. * Update opentelemetry-ebpf-profiler with latest changes from upstream. * Update 3rdparty licenses. * backport: Refactor symbol caching open-telemetry/opentelemetry-ebpf-profiler#635 * Use containerID provided by eBPF profiler when available and split by service is enabled. * Do not collect Go labels by default

#### Description Update go.opentelemetry.io/ebpf-profiler to use a tagged version instead of a commit. As open-telemetry/opentelemetry-ebpf-profiler#635 removed libpf.FrameID, also move its functionality to this package as it is essential for its functionality.  #### Link to tracking issue Fixes  #### Testing  #### Documentation  --------- Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

fabled added 3 commits July 20, 2025 19:38

lint and test suite fixes

13b2ffb

more lint and testing fixes

c490b0a

fabled marked this pull request as ready for review July 20, 2025 17:20

fabled requested review from a team as code owners July 20, 2025 17:20

fabled added 3 commits July 21, 2025 08:34

remove reporter.Config.FramesCacheElements

0933e3a

restore ruby first level cache

f65997b

I removed this as I was planning to include the tracehandler cache changes making it redundant, but as it is wort a separate PR lets add this back for now.

remove unused pdata.FrameMapLifetime

d4074cb

fabled mentioned this pull request Jul 21, 2025

Executable metadata missing for some executables on hosts with a large number of executables #250

Closed

Merge branch 'main' into tt-refactor-symbolization-cache

dd4545b

christos68k reviewed Jul 28, 2025

View reviewed changes

Comment thread libpf/trace.go Outdated

Comment thread libpf/trace.go Outdated

Comment thread libpf/trace.go Outdated

address review feedback

cce62b1

florianl reviewed Jul 29, 2025

View reviewed changes

christos68k approved these changes Jul 29, 2025

View reviewed changes

Comment thread interpreter/types.go Outdated

Comment thread interpreter/types.go Outdated

florianl approved these changes Jul 30, 2025

View reviewed changes

fabled added 2 commits July 30, 2025 17:42

Merge branch 'main' into tt-refactor-symbolization-cache

602f71c

address review feedback

eeafc91

fabled merged commit 17e31d8 into open-telemetry:main Jul 30, 2025
27 checks passed

nsavoire added a commit to DataDog/dd-otel-host-profiler that referenced this pull request Aug 4, 2025

backport: Refactor symbol caching

dec072f

open-telemetry/opentelemetry-ebpf-profiler#635

nsavoire added a commit to DataDog/dd-otel-host-profiler that referenced this pull request Aug 7, 2025

backport: Refactor symbol caching

1ceafd6

open-telemetry/opentelemetry-ebpf-profiler#635

nsavoire added a commit to DataDog/dd-otel-host-profiler that referenced this pull request Aug 8, 2025

backport: Refactor symbol caching

fab8e5a

open-telemetry/opentelemetry-ebpf-profiler#635

florianl mentioned this pull request Aug 14, 2025

[chore] Update go.opentelemetry.io/ebpf-profiler to v0.0.202531 open-telemetry/opentelemetry-collector-contrib#42008

Merged

gnurizen mentioned this pull request Aug 14, 2025

fix golabels tests parca-dev/opentelemetry-ebpf-profiler#92

Closed

nsavoire added a commit to DataDog/dd-otel-host-profiler that referenced this pull request Aug 18, 2025

backport: Refactor symbol caching

d0e64cd

open-telemetry/opentelemetry-ebpf-profiler#635

gnurizen mentioned this pull request Aug 18, 2025

fix golabels test parca-dev/opentelemetry-ebpf-profiler#97

Closed

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 16, 2025

Refactor symbol caching (open-telemetry#635)

1a41a64

gnurizen mentioned this pull request Sep 16, 2025

pr635 parca-dev/opentelemetry-ebpf-profiler#106

Merged

gnurizen pushed a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 17, 2025

Refactor symbol caching (open-telemetry#635)

229c304

gnurizen mentioned this pull request Mar 9, 2026

Optimize distro QEMU tests to be more efficient parca-dev/opentelemetry-ebpf-profiler#228

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor symbol caching#635

Refactor symbol caching#635
fabled merged 10 commits intoopen-telemetry:mainfrom
fabled:tt-refactor-symbolization-cache

fabled commented Jul 20, 2025 •

edited

Loading

Uh oh!

christos68k left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

florianl left a comment

Uh oh!

florianl Jul 29, 2025

Uh oh!

fabled Jul 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

fabled commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christos68k left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

florianl left a comment

Choose a reason for hiding this comment

Uh oh!

florianl Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

fabled Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fabled commented Jul 20, 2025 •

edited

Loading