python: get the Thread State from a Thread-local by florianl · Pull Request #1109 · open-telemetry/opentelemetry-ebpf-profiler

florianl · 2026-01-23T10:41:26Z

While looking into #1054 I noticed that python unwinding fails starting for python 3.13 on ARM64.

Starting from Python 3.13, internals changed with python/cpython#103323.

This change originates from python/cpython#103323. Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

fabled · 2026-01-26T08:28:02Z

+// extractTLSOffsetFromCodeAMD64 extracts the TLS offset by analyzing x86_64 assembly code.
+// It looks for MOV instructions with FS segment prefix (e.g., MOV rax, FS:[offset]).
+func extractTLSOffsetFromCodeAMD64(code []byte, baseAddr uint64) (int64, error) {


This function looks like a partial copy of

opentelemetry-ebpf-profiler/interpreter/golabels/tls_amd64.go

Line 24 in 7bc54bc

func extractTLSGOffset(f *pfelf.File) (int32, error) {

I think it would make sense to add the common portion of finding FS:xxx and resolving the xxx to asm/amd as a helper function. The other function also supports RIP relative stuff, so using that as reference would likely be better.

I have added asm/amd/ExtractFSOffsetFromCode() with d08582d.

While I knew about the duplication with golabels, I wasn't sure how my approach will be received and the difference to resolving the memory reference (Pattern 3). Let me know if this deduplication works for you.

fabled · 2026-01-26T08:38:30Z

+// extractTLSOffsetFromCodeARM64 extracts the TLS offset by analyzing ARM64 assembly code.
+// It looks for the pattern: MRS Xn, TPIDR_EL0 followed by ADD Xn, Xn, #offset or LDR [Xn, #offset].
+func extractTLSOffsetFromCodeARM64(code []byte, baseAddr uint64, visited map[uint64]bool, depth int, ef *pfelf.File) (int64, error) {


Immediately looking, this probably is generic code and could also live in asm/arm as helper?

Also if we stick with a recursive function instead of iteration, we can hide it as either an inner function or a separate recursive function that's called by the wrapper (which shouldn't take a map argument). Otherwise, it's sort of ugly to leak internal implementation details (allocate and pass a map to carry state across recursive calls) to the caller.

Moved and refactored the code with 72de6e3.
Hope this works for you.

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

fabled

looks pretty good to me. some minor clean up comments added

Co-authored-by: Timo Teräs <timo.teras@iki.fi>

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

fabled

lgtm. thanks!

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

#1109 introduced a new asm/arm package. Move functionality of armhelpers into this new package for consistency. Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

bobrik · 2026-02-04T02:47:08Z

I just saw this on x86_64:

Feb 04 02:43:59 748m17 ebpf-profiler[1492724]: time=2026-02-04T02:43:59.480Z level=WARN msg="Failed to extract TLS offset: could not extract TLS offset from _PyThreadState_GetCurrent: could not find FS-relative MOV instruction with valid TLS offset"

It seems related to this PR.

PR open-telemetry#1109 already handles Python 3.13+ TLS access via staticTLSOffset, making these checks unnecessary. The eBPF code checks tls_offset directly and uses it for Python 3.13+ without requiring TSDInfo.

When LibcInfo is collected from multiple DSOs (e.g., libc.so and ld-linux.so), UpdateLibcInfo may be called multiple times. For Python versions that don't have a static TLS offset (< 3.13 or when extraction fails), we need TSDInfo to access thread state. Wait until TSDInfo is available before inserting proc data, and prevent duplicate inserts. This is a simplified version that checks staticTLSOffset directly rather than version numbers, since PR open-telemetry#1109 already extracts the TLS offset for Python 3.13+ when available.

For statically-linked Ruby binaries (bin/ruby rather than libruby.so), TLS descriptors are not available. Instead, rb_current_ec_noinline accesses the execution context directly via a TP-relative offset (FS:offset on x86_64, MRS tpidr_el0 + ADD on aarch64). This reuses the asm/amd.ExtractTLSOffset and asm/arm.ExtractTLSOffset infrastructure from the Python TLS PR (open-telemetry#1109) to disassemble rb_current_ec_noinline and extract the offset. Also changes current_ec_tpbase_tls_offset from u64 to s64 since static TLS offsets (local exec model) are negative on x86_64.

For statically-linked Ruby binaries (bin/ruby rather than libruby.so), TLS descriptors are not available. Instead, rb_current_ec_noinline accesses the execution context directly via a TP-relative offset (FS:offset on x86_64, MRS tpidr_el0 + ADD on aarch64). This reuses the asm/amd.ExtractTLSOffset and asm/arm.ExtractTLSOffset infrastructure from the Python TLS PR (open-telemetry#1109) to disassemble rb_current_ec_noinline and extract the offset. Also changes current_ec_tpbase_tls_offset from u64 to s64 since static TLS offsets (local exec model) are negative on x86_64. Also adds RUBY_DISABLE_GC env var support to the loop.rb test script to allow capturing coredumps without GC interference.

florianl added the interpreter/python label Jan 23, 2026

florianl requested review from a team as code owners January 23, 2026 10:41

florianl marked this pull request as draft January 23, 2026 10:44

florianl force-pushed the python-tls branch from 0caf197 to 4bdc15b Compare January 23, 2026 10:55

python: get the Thread State from a Thread-local

022c775

This change originates from python/cpython#103323. Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

florianl force-pushed the python-tls branch from 4bdc15b to 022c775 Compare January 23, 2026 10:59

florianl marked this pull request as ready for review January 23, 2026 11:03

florianl added the bug Something isn't working label Jan 23, 2026

fabled reviewed Jan 26, 2026

View reviewed changes

florianl and others added 9 commits January 27, 2026 10:17

Merge branch 'main' into python-tls

9242180

replace s8 with s16

f70d901

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

simplify get_PyThreadState check

6379111

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

make format-ebpf

5c50c99

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

drop duplicate check

4ae3bdd

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

add ExtractFSOffsetFromCode

d08582d

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

create package asm/arm

72de6e3

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

Merge branch 'main' into python-tls

021d9de

Merge branch 'main' into python-tls

6bbb6dc

florianl requested review from christos68k and fabled January 29, 2026 13:16

fabled reviewed Jan 29, 2026

View reviewed changes

Comment thread asm/arm/tls.go Outdated

Comment thread asm/amd/tls.go Outdated

Comment thread interpreter/python/amd64_decode.go Outdated

florianl and others added 3 commits January 29, 2026 15:54

Apply suggestions from code review

9589712

Co-authored-by: Timo Teräs <timo.teras@iki.fi>

apply changes

09ffc40

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

move TLS offset check

dc4bedd

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

fabled approved these changes Jan 29, 2026

View reviewed changes

christos68k reviewed Jan 30, 2026

View reviewed changes

Comment thread asm/amd/tls.go Outdated

Comment thread asm/amd/tls.go Outdated

Comment thread asm/arm/tls.go Outdated

Comment thread asm/arm/tls.go Outdated

Comment thread asm/arm/tls.go Outdated

Comment thread interpreter/python/python.go Outdated

florianl added 2 commits January 30, 2026 10:11

Merge branch 'main' into python-tls

0fb11c9

apply feedback

1b382a4

Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

christos68k approved these changes Jan 30, 2026

View reviewed changes

christos68k merged commit 619e4c1 into open-telemetry:main Jan 30, 2026
32 checks passed

florianl added a commit that referenced this pull request Jan 30, 2026

asm/arm: merge armhelpers

1501ab9

#1109 introduced a new asm/arm package. Move functionality of armhelpers into this new package for consistency. Signed-off-by: Florian Lehner <florian.lehner@elastic.co>

florianl mentioned this pull request Jan 30, 2026

asm/arm: merge with armhelpers #1143

Merged

florianl mentioned this pull request Feb 4, 2026

Extract DTV info from __tls_get_addr, add to LibcInfo #929

Merged

dalehamel mentioned this pull request Mar 2, 2026

ruby: support EC offset extraction for statically-linked Ruby Shopify/opentelemetry-ebpf-profiler#28

Closed

dalehamel mentioned this pull request Mar 4, 2026

ruby: support EC offset extraction for statically-linked Ruby #1227

Merged

gnurizen mentioned this pull request Mar 9, 2026

Optimize distro QEMU tests to be more efficient parca-dev/opentelemetry-ebpf-profiler#228

Closed

Conversation

florianl commented Jan 23, 2026

Uh oh!

Uh oh!

fabled Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

florianl Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fabled Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

christos68k Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

florianl Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

fabled left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fabled left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bobrik commented Feb 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

christos68k Jan 27, 2026 •

edited

Loading