Skip to content

interpreter: Support multiple interpreters for single ELF object#702

Merged
christos68k merged 7 commits intoopen-telemetry:mainfrom
parca-dev:multi-interpreters
Aug 19, 2025
Merged

interpreter: Support multiple interpreters for single ELF object#702
christos68k merged 7 commits intoopen-telemetry:mainfrom
parca-dev:multi-interpreters

Conversation

@gnurizen
Copy link
Copy Markdown
Contributor

@gnurizen gnurizen commented Aug 14, 2025

  • Add MultiData and MultiInstance types to handle multiple interpreters
  • Modify detectAndLoadInterpData to try all loaders instead of stopping at first match
  • Use errors.Join for proper compound error handling
  • Enables concurrent use of Go tracer and GoLabels on same binary

Fixes issue where only the first matching interpreter loader would be used,
preventing multiple interpreters from handling the same ELF object.

Fixes #687

@gnurizen gnurizen requested review from a team as code owners August 14, 2025 15:34
@gnurizen gnurizen force-pushed the multi-interpreters branch 2 times, most recently from 4a0f12a to a7e9f0b Compare August 14, 2025 17:09
Copy link
Copy Markdown
Contributor

@fabled fabled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good! Thanks! Few changes requested to MultiInstance.Symbolize implementation.

Comment thread interpreter/multi.go Outdated
Comment on lines +119 to +130
if err == nil && len(*frames) > initialLen {
// Successfully symbolized by this interpreter
return nil
}
// If symbolization didn't add any frames, reset and try next interpreter
if len(*frames) == initialLen {
continue
}
// If frames were added but there was an error, keep them and return
if err != nil {
return err
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if err == nil && len(*frames) > initialLen {
// Successfully symbolized by this interpreter
return nil
}
// If symbolization didn't add any frames, reset and try next interpreter
if len(*frames) == initialLen {
continue
}
// If frames were added but there was an error, keep them and return
if err != nil {
return err
}
if err != ErrMismatchInterpreterType {
return err
}

The contract for Symbolizehas been to return interpreter.ErrMismatchInterpreterType if it did not recognize the frame. I think we can just return err here if it does not match this, because then this host.Frame belonged to this interpreter and something went wrong, or it successfully handled the frame.

Comment thread interpreter/multi.go Outdated
return err
}
}
return nil
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return nil
return ErrMismatchInterpreterType

If none of the interpreters succeeds to handle the frame, this function should report this as an error.

- Add MultiData and MultiInstance types to handle multiple interpreters
- Modify detectAndLoadInterpData to try all loaders instead of stopping at first match
- Use errors.Join for proper compound error handling
- Enables concurrent use of Go tracer and GoLabels on same binary

Fixes issue where only the first matching interpreter loader would be used,
preventing multiple interpreters from handling the same ELF object.

Fixes open-telemetry#687
@gnurizen gnurizen force-pushed the multi-interpreters branch from a7e9f0b to c4ed413 Compare August 18, 2025 17:59
@gnurizen
Copy link
Copy Markdown
Contributor Author

Thanks for the review @fabled, should be good to go!

Copy link
Copy Markdown
Contributor

@fabled fabled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

Comment thread interpreter/multi.go
allMetrics = append(allMetrics, metrics...)
}

return allMetrics, errors.Join(errs...)
Copy link
Copy Markdown
Member

@christos68k christos68k Aug 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks the existing caller assumption that no useful metrics are returned if error != nil, so we should probably also update the caller to process the metrics regardless of error.

Comment thread interpreter/multi.go
// This is a valid state - return nil, nil
return nil, nil
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're potentially swallowing/hiding an error here (if len(instances) > 0 && len(errs) > 0)), maybe we should log the latter case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch

Comment thread processmanager/manager.go Outdated
Comment thread processmanager/manager.go
}

// Update metrics even if there was an error, because its possible ii is a multi-instance
// and some of the instances may have returned metrics.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// and some of the instances may have returned metrics.
// and some of the underlying instances may have returned metrics.

Comment thread interpreter/multi.go
Comment thread interpreter/multi.go Outdated
@christos68k
Copy link
Copy Markdown
Member

Thanks, added some nits and simplifications, LGTM otherwise!

gnurizen and others added 4 commits August 19, 2025 12:02
Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
@christos68k christos68k merged commit 0e3b5bb into open-telemetry:main Aug 19, 2025
27 of 28 checks passed
@brancz brancz deleted the multi-interpreters branch August 20, 2025 11:37
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 8, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 8, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
gnurizen added a commit to parca-dev/opentelemetry-ebpf-profiler that referenced this pull request Sep 18, 2025
…n-telemetry#702)

Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Golabels Never Activated?

3 participants