Skip to content

Extract DTV info from __tls_get_addr, add to LibcInfo#929

Merged
fabled merged 19 commits into
open-telemetry:mainfrom
Shopify:libcinfo-tsd-tls-refactor
Mar 3, 2026
Merged

Extract DTV info from __tls_get_addr, add to LibcInfo#929
fabled merged 19 commits into
open-telemetry:mainfrom
Shopify:libcinfo-tsd-tls-refactor

Conversation

@dalehamel
Copy link
Copy Markdown
Contributor

@dalehamel dalehamel commented Nov 5, 2025

What

Refactor's tpbase to be libc package, as it provides additional info about libc.

In particular, this adds DTV introspection information and bundles this with the TSD info into a new LibcInfo.

Addresses #883 for all cases except static TLS.

Why

We need the DTV information to look up TLS variables when TLS descriptors are not available.

Refactoring to a generic "libc" approach is something @fabled suggested.

How

This adds disassembler code for extracting the DTV information from __tls_get_addr if it is present in the libc.

It should be supported for musl and glibc, on both x86_64 and aarch64, and test cases from different libc versions and architectures are added to validate that this is the case.

The tpbase package is renamed to libc, and related functions are renamed to be more generic, but otherwise the functionality is unchanged.

Nothing actually uses the new DTV data yet, another PR will follow up to add that for Ruby which reads the execution context from a TLS variable.

@dalehamel dalehamel requested review from a team as code owners November 5, 2025 14:29
Comment thread libc/libc_test.go
Comment thread libc/libc_aarch64.go Fixed
Comment thread libc/libc_aarch64.go Fixed
Comment thread libc/libc_test.go
Comment thread libc/libc_x86.go
Comment thread libc/libc_x86.go
}

// extractDTVInfoX86 analyzes __tls_get_addr to find the DTV offset from FS base
func extractDTVInfoX86(code []byte) (DTVInfo, error) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the actual new functionality.

Comment thread libc/libc.go
Comment thread libc/libc.go
// extractDTVInfo extracts the introspection data for the DTV to access TLS vars
func extractDTVInfo(ef *pfelf.File) (*DTVInfo, error) {
var info DTVInfo
_, code, err := ef.SymbolData("__tls_get_addr", 2048)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the test coredumps don't actually have this symbol, so it is necessary to not error out if the symbol isn't present. Hence why i added the logging package, and we now just log the error if the symbol is missing.

We return an empty DTVInfo struct, it is up to users of DTV info to check that it is valid before using it. This can easily be done by verifying that "EntryWidth" is not 0.

In the cases where we DO have the symbol, but fail to extract info from it, we legitimately error out.

Copy link
Copy Markdown
Contributor

@fabled fabled Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this is defined in the ld-linux-x86-64.so.2 in (some versions of) glibc. So it means that the libc information may need to be collected from two DSOs in case of glibc.

You should add this to the regexp pattern in IsPotentialTSDDSO. Perhaps rename that to IsLibcDSO?

This also means that ProcessManager.assignLibcInfo should be updated to merge the information from these two different DSOs. Probably add a helper libc.MergeLibcInfo or add a struct method for LibcInfo.Merge?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the ability to check for equality and merge, and the values are "accumulated" when we call assignLibcInfo.

Also added unit tests for the associated LibcInfo.IsEqual and LibcInfo.Merge, and to verify the accumulation behaviour in assignLibcInfo.

@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch 2 times, most recently from 1abf5f4 to 91a89bc Compare November 5, 2025 14:42
Comment thread libc/libc_aarch64.go Fixed
Copy link
Copy Markdown
Contributor

@fabled fabled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Some comments added. And I see that unit tests are missing for the new TLS extraction code.

If not too much work, could the mechanical conversion from TSD -> libc on the hook/struct be done a separate PR first? And then add the new functionality along with the missing tests in a next PR? I think the mechanical work should be trivial to get reviewed and merged first.

Comment thread libc/libc.go Outdated
Comment thread libc/libc.go Outdated
Comment thread libc/libc.go Outdated
Comment thread libc/libc_aarch64.go Outdated
switch inst.Op {

case aa.MRS:
foundThreadPtr = true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The register to which the value is loaded is not tracked? Also there can be other MRS variants, should also check the actual full opcode like done in extractTSDInfoARM.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed to bring it more in-line with extractTSDInfoARM, also fixed a test that turned out to be extracting the wrong value which this caught.

Comment thread libc/libc_x86.go Outdated
Comment thread processmanager/processinfo.go Outdated
@dalehamel
Copy link
Copy Markdown
Contributor Author

Thanks for working on this! Some comments added.

Great comments, will address them.

And I see that unit tests are missing for the new TLS extraction code.

Actually it is there, just the renaming of files causes it to be hidden by default. Unless you are referring to additional unit tests you wanted?

If not too much work, could the mechanical conversion from TSD -> libc on the hook/struct be done a separate PR first? And then add the new functionality along with the missing tests in a next PR? I think the mechanical work should be trivial to get reviewed and merged first.

I'll do this first, that should make this PR a lot easier to review once it is rebased on that. Git gets really confused about the additions happening as well as the renames.

@dalehamel
Copy link
Copy Markdown
Contributor Author

If not too much work, could the mechanical conversion from TSD -> libc on the hook/struct be done a separate PR first? And then add the new functionality along with the missing tests in a next PR? I think the mechanical work should be trivial to get reviewed and merged first.

Done in #952

@dalehamel dalehamel marked this pull request as draft November 10, 2025 19:20
@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch from 5cb9b33 to 23da4c0 Compare November 11, 2025 14:49
@dalehamel
Copy link
Copy Markdown
Contributor Author

I'll wait for #956 and rebase this again before reopening for review

@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch from 23da4c0 to f2bede8 Compare November 11, 2025 14:51
@dalehamel dalehamel changed the title Refactor tpbase to 'libc', and add DTV disasm Extract DTV info from __tls_get_addr, add to LibcInfo Nov 11, 2025
@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch from f2bede8 to 973a3ce Compare November 12, 2025 14:00
Comment thread libc/libc.go Outdated
@dalehamel
Copy link
Copy Markdown
Contributor Author

FYI this is next on my to-do list, not ready for another review yet still have comments to address

@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch from 99e5108 to a437cf3 Compare November 21, 2025 15:18
Comment thread libc/libc_aarch64.go Fixed
@dalehamel dalehamel force-pushed the libcinfo-tsd-tls-refactor branch 2 times, most recently from 470f89b to acf2e8f Compare November 21, 2025 15:29
Copy link
Copy Markdown
Member

@florianl florianl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay and thanks for the work 🙏

@florianl florianl requested a review from fabled February 26, 2026 07:26
@dalehamel
Copy link
Copy Markdown
Contributor Author

Sorry for the delay and thanks for the work 🙏

Cheers, much appreciated. Once we can get this landed i have some ruby EC related fixes to submit which will build on it :)

Copy link
Copy Markdown
Contributor

@fabled fabled left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 2, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 2, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
@fabled fabled merged commit 63d24dc into open-telemetry:main Mar 3, 2026
35 checks passed
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 3, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 3, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 4, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 9, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 19, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 24, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 24, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Mar 27, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to dalehamel/opentelemetry-ebpf-profiler that referenced this pull request Apr 1, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to dalehamel/opentelemetry-ebpf-profiler that referenced this pull request Apr 1, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
dalehamel added a commit to Shopify/opentelemetry-ebpf-profiler that referenced this pull request Apr 7, 2026
When TLSDESC relocations are unavailable (e.g. some musl setups or
dynamically allocated TLS), the Ruby execution context can still be
found by traversing the Dynamic Thread Vector (DTV).

This commit:

- Adds a shared dtv_read() helper in tsd.h (alongside existing tsd_read)
  that traverses the DTV using DTVInfo from PR open-telemetry#929's libc introspection.
  This helper is available to all interpreters, not just Ruby.

- Generalizes VisitTLSRelocations into VisitRelocations with a pluggable
  relocation type filter, enabling lookup of DTPMOD64 relocations to find
  the TLS module ID offset for libruby.so.

- Adds DTVInfo, current_ec_tls_offset, and tls_module_id fields to
  RubyProcInfo so the eBPF unwinder can use DTV traversal.

- Implements UpdateLibcInfo for Ruby to receive DTVInfo when the libc
  package provides it (may arrive from a separate DSO like ld-linux.so).

- Adds a DTV fallback path in ruby_tracer.ebpf.c between the existing
  TLSDESC path and the ractor fallback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants