Add experimental C/C++ implementation for process context OTEP by ivoanjo · Pull Request #23 · open-telemetry/sig-profiling

ivoanjo · 2025-12-01T16:44:23Z

This PR adds an experimental C/C++ implementation for the "Process Context" OTEP being proposed in open-telemetry/opentelemetry-specification#4719

This implementation previously lived in https://github.com/ivoanjo/proc-level-demo/tree/main/anonmapping-clib and as discussed during the OTEL profiling SIG meeting we want to add it to this repository so it becomes easier to find and contribute to.

I've made sure to include a README explaining how to use it. Here's the ultra-quick start (Linux-only):

$ ./build.sh
$ ./build/example_ctx --keep-running
Published: service=my-service, instance=123d8444-2c7e-46e3-89f6-6217880f7123, env=prod, version=4.5.6, sdk=example_ctx.c/c/1.2.3, resources=resource.key1=resource.value1,resource.key2=resource.value2
Continuing forever, to exit press ctrl+c...
TIP: You can now `sudo ./otel_process_ctx_dump.sh 267023` to see the context

 # In another shell
$ sudo ./otel_process_ctx_dump.sh 267023 # Update this to match the PID from above
Found OTEL context for PID 267023
Start address: 756f28ce1000
00000000  4f 54 45 4c 5f 43 54 58  02 00 00 00 0b 68 55 47  |OTEL_CTX.....hUG|
00000010  70 24 7d 18 50 01 00 00  a0 82 6d 7e 6a 5f 00 00  |p$}.P.....m~j_..|
00000020
Parsed struct:
  otel_process_ctx_signature       : "OTEL_CTX"
  otel_process_ctx_version         : 2
  otel_process_ctx_published_at_ns : 1764606693650819083 (2025-12-01 16:31:33 GMT)
  otel_process_payload_size        : 336
  otel_process_payload             : 0x00005f6a7e6d82a0
Payload dump (336 bytes):
00000000  0a 25 0a 1b 64 65 70 6c  6f 79 6d 65 6e 74 2e 65  |.%..deployment.e|
00000010  6e 76 69 72 6f 6e 6d 65  6e 74 2e 6e 61 6d 65 12  |nvironment.name.|
...
Protobuf decode:
attributes {
  key: "deployment.environment.name"
  value {
    string_value: "prod"
  }
}
attributes {
  key: "service.instance.id"
  value {
    string_value: "123d8444-2c7e-46e3-89f6-6217880f7123"
  }
}
attributes {
  key: "service.name"
  value {
    string_value: "my-service"
  }
}
...

Note that because the upstream OTEP is still under discussion, this implementation is experimental and may need changes to match up with the final version of the OTEP.

This PR adds an experimental C/C++ implementation for the "Process Context" OTEP being proposed in open-telemetry/opentelemetry-specification#4719 This implementation previously lived in https://github.com/ivoanjo/proc-level-demo/tree/main/anonmapping-clib and as discussed during the OTEL profiling SIG meeting we want to add it to this repository so it becomes easier to find and contribute to. I've made sure to include a README explaining how to use it. Here's the ultra-quick start (Linux-only): ```bash $ ./build.sh $ ./build/example_ctx --keep-running Published: service=my-service, instance=123d8444-2c7e-46e3-89f6-6217880f7123, env=prod, version=4.5.6, sdk=example_ctx.c/c/1.2.3, resources=resource.key1=resource.value1,resource.key2=resource.value2 Continuing forever, to exit press ctrl+c... TIP: You can now `sudo ./otel_process_ctx_dump.sh 267023` to see the context # In another shell $ sudo ./otel_process_ctx_dump.sh 267023 # Update this to match the PID from above Found OTEL context for PID 267023 Start address: 756f28ce1000 00000000 4f 54 45 4c 5f 43 54 58 02 00 00 00 0b 68 55 47 |OTEL_CTX.....hUG| 00000010 70 24 7d 18 50 01 00 00 a0 82 6d 7e 6a 5f 00 00 |p$}.P.....m~j_..| 00000020 Parsed struct: otel_process_ctx_signature : "OTEL_CTX" otel_process_ctx_version : 2 otel_process_ctx_published_at_ns : 1764606693650819083 (2025-12-01 16:31:33 GMT) otel_process_payload_size : 336 otel_process_payload : 0x00005f6a7e6d82a0 Payload dump (336 bytes): 00000000 0a 25 0a 1b 64 65 70 6c 6f 79 6d 65 6e 74 2e 65 |.%..deployment.e| 00000010 6e 76 69 72 6f 6e 6d 65 6e 74 2e 6e 61 6d 65 12 |nvironment.name.| ... Protobuf decode: attributes { key: "deployment.environment.name" value { string_value: "prod" } } attributes { key: "service.instance.id" value { string_value: "123d8444-2c7e-46e3-89f6-6217880f7123" } } attributes { key: "service.name" value { string_value: "my-service" } } ... ``` Note that because the upstream OTEP is still under discussion, this implementation is experimental and may need changes to match up with the final version of the OTEP.

(we may need to adjust the license further at some point)

christos68k

After building, on 6.12.9-amd64 execution fails as the mapping can't be found. I added an fprintf(stderr, "%s", line); inside the try_finding_mapping loop, you can see the output below:

sig-profiling/process-context/c-and-cpp$ ./build/example_ctx --keep-running
55a54d38c000-55a54d38d000 r--p 00000000 08:01 9709685                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/example_ctx
55a54d38d000-55a54d38e000 r-xp 00001000 08:01 9709685                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/example_ctx
55a54d38e000-55a54d38f000 r--p 00002000 08:01 9709685                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/example_ctx
55a54d38f000-55a54d390000 r--p 00002000 08:01 9709685                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/example_ctx
55a54d390000-55a54d391000 rw-p 00003000 08:01 9709685                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/example_ctx
55a55c371000-55a55c392000 rw-p 00000000 00:00 0                          [heap]
7f701c699000-7f701c69c000 rw-p 00000000 00:00 0 
7f701c69c000-7f701c6c4000 r--p 00000000 08:01 568749                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f701c6c4000-7f701c829000 r-xp 00028000 08:01 568749                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f701c829000-7f701c87f000 r--p 0018d000 08:01 568749                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f701c87f000-7f701c883000 r--p 001e2000 08:01 568749                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f701c883000-7f701c885000 rw-p 001e6000 08:01 568749                     /usr/lib/x86_64-linux-gnu/libc.so.6
7f701c885000-7f701c892000 rw-p 00000000 00:00 0 
7f701c8a9000-7f701c8ab000 r--p 00000000 00:00 0 
7f701c8ab000-7f701c8ac000 r--p 00000000 08:01 9709667                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/libotel_process_ctx.so
7f701c8ac000-7f701c8ae000 r-xp 00001000 08:01 9709667                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/libotel_process_ctx.so
7f701c8ae000-7f701c8af000 r--p 00003000 08:01 9709667                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/libotel_process_ctx.so
7f701c8af000-7f701c8b0000 r--p 00003000 08:01 9709667                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/libotel_process_ctx.so
7f701c8b0000-7f701c8b1000 rw-p 00004000 08:01 9709667                    /home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/build/libotel_process_ctx.so
7f701c8b1000-7f701c8b3000 rw-p 00000000 00:00 0 
7f701c8b3000-7f701c8b7000 r--p 00000000 00:00 0                          [vvar]
7f701c8b7000-7f701c8b9000 r-xp 00000000 00:00 0                          [vdso]
7f701c8b9000-7f701c8ba000 r--p 00000000 08:01 568746                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f701c8ba000-7f701c8e2000 r-xp 00001000 08:01 568746                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f701c8e2000-7f701c8ed000 r--p 00029000 08:01 568746                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f701c8ed000-7f701c8ef000 r--p 00034000 08:01 568746                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f701c8ef000-7f701c8f0000 rw-p 00036000 08:01 568746                     /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f701c8f0000-7f701c8f1000 rw-p 00000000 00:00 0 
7ffce698e000-7ffce69af000 rw-p 00000000 00:00 0                          [stack]
Failed to read context: No OTEL_CTX mapping found (/home/chris/code/_work/open-telemetry/sig-profiling/process-context/c-and-cpp/otel_process_ctx.c:590)

…ld directory

ivoanjo · 2025-12-05T14:09:13Z

After building, on 6.12.9-amd64 execution fails as the mapping can't be found.

That's quite unexpected!

I suspect the context was actually published, but the naming failed...
In the /proc/pid/maps you shared there's

7f701c8a9000-7f701c8ab000 r--p 00000000 00:00 0

which is two pages long (8192 bytes) and read-only so I suspect that's our little corner.

I'm assuming you saw no errors, and every system call we're doing has extensive error checking.... other than naming the mapping... So it points in that direction too.

Can you try hardcoding named_mapping_supported() to false in the file? Right now the reading code never tries to fallback to probing instead of using the name if you're on a newer kernel, but if it was only the naming that failed then that should work.

A similar trick should also work with otel_process_ctx_dump.sh -- tweaking the check for kernel version on line 34 to always force the legacy codepath should also make that work if this is the issue.

So the question is: why, on kernel 6.12.9, is the naming not working?

Ubuntu keeps builds of most kernel versions but they didn't have 6.12.9, so I tried on 6.12.4 and 6.12.2:

$ uname -a
Linux marumitwice 6.12.4-061204-generic #202412091239 SMP PREEMPT_DYNAMIC Fri Feb 14 00:41:41 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
$ ./build/example_ctx --keep-running &
[1] 5907
Published: service=my-service, instance=123d8444-2c7e-46e3-89f6-6217880f7123, env=prod, version=4.5.6, sdk=example_ctx.c/c/1.2.3, resources=resource.key1=resource.value1,resource.key2=resource.value2
Continuing forever, to exit press ctrl+c...
TIP: You can now `sudo ./otel_process_ctx_dump.sh 5907` to see the context
$ cat /proc/5907/maps | grep OTEL
7af6b3cf4000-7af6b3cf6000 r--p 00000000 00:00 0                          [anon:OTEL_CTX]

(same result for 6.12.12). So... I don't think it's a kernel thing. Do you mind trying to run sudo strace build/example_ctx and sharing the result? I'm curious what we'll see happen around the naming syscall.

christos68k · 2025-12-05T14:26:15Z

(same result for 6.12.12). So... I don't think it's a kernel thing. Do you mind trying to run sudo strace build/example_ctx and sharing the result? I'm curious what we'll see happen around the naming syscall.

prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, 0x7f6f26504000, 8192, "OTEL_CTX") = -1 EINVAL (Invalid argument)

ivoanjo · 2025-12-05T14:31:01Z

Cool, let me investigate a bit and report back!

christos68k · 2025-12-05T14:31:51Z

Cool, let me investigate a bit and report back!

OK so it seems kernel version 5.17+ is not enough, it also needs to be enabled in the kernel config (and in my Debian install, it's not. Ubuntu does enable it by default however):

$ grep -i CONFIG_ANON_VMA_NAME /boot/config-6.12.9-amd64 
# CONFIG_ANON_VMA_NAME is not set

ivoanjo · 2025-12-05T14:39:27Z

Yup, that would be it! It looks like it's a mix -- looking at the upstream kernel.org for 6.18 this is not enabled by default (other than for s/390); but it looks like many vendors do enable it by default: raspberrypi/linux#7008 (comment) .

This being the case, I'll go and update the implementation to stop assuming that this will always work on 5.17+ -- it can always fail, and in that case, the nameless fallback will pick up the slack.

As uncovered during review, it's possible that even on modern kernels, we're not able to name the process context mapping. This is not a big problem since we already had a fallback for older kernels, so I've tweaked the code so that the fallback is also usable on modern kernels (e.g. no more peeking at kernel version when deciding to use the fallback or not). This doesn't impact publishing (it already accounted for naming possibly failing), but I've updated the testing reader inside `otel_process_ctx.c` as well as the one in `otel_process_ctx_dump.sh`.

ivoanjo · 2025-12-05T15:15:56Z

Thanks for flagging this one! I've pushed 38bf9fe to update the reader code to fallback to working without naming whenever the name isn't found.

This should make the context work out of the box for you @christos68k

ivoanjo · 2025-12-05T15:16:50Z

(I'll update the spec next to reflect this -- I think there's a sentence or two there about the naming)

christos68k

LGTM thanks!

christos68k · 2025-12-05T16:42:48Z

+  time_t t0 = time(NULL);
+  while (time(NULL) - t0 < seconds) {
+    x += burn_cpu();
+  }


Nit: Maybe add a getchar() -optionally- here to allow for manual control of updates (so that I can execute the dump script to verify values without rushing to meet a time limit)

👍 Added in ca75c64

Add pid when printing context in example Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>

…n Linux 5.17+ See open-telemetry/sig-profiling#23 for a wider discussion of this.

See also open-telemetry/opentelemetry-specification#4719 (comment)

felixge

LGTM. Thanks!

…h memfd + in-place modification (#34) This PR updates the experimental C/C++ implementation for the "Process Context OTEP" introduced in #23 and being proposed in open-telemetry/opentelemetry-specification#4719 with the latest updates from the upstream discussion -- open-telemetry/opentelemetry-specification@3caecfb . Specifically: * Instead of always using an anonymous mapping, try first to create a memfd and create a mapping from the memfd. If due to security restrictions memfd is not available, fall back to an anonymous mapping instead. * Remove probing as a fallback for when naming a mapping fails. Because the name of a memfd also shows up in `/proc/<pid>/maps`, we expect that having `memfd` naming as a fallback for when `prctl` is not available is enough. * Drop 2-page size and read-only permissions on the header memory pages. These were intended to support the "probing as a fallback for naming failure", so they are no longer needed. * Introduce in-place updates to process context. This allows efficient updates. In particular, it makes it easier for the reader to detect updates and avoids reparsing `/proc/<pid>/maps` for updates. --------- Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>

ivoanjo requested a review from a team as a code owner December 1, 2025 16:44

Remove incorrect licence claim

524867d

(we may need to adjust the license further at some point)

christos68k reviewed Dec 5, 2025

View reviewed changes

Comment thread process-context/c-and-cpp/otel_process_ctx.cpp Outdated

Avoid need for .cpp copy of the file, create on demand on cmake bui…

af39cc9

…ld directory

christos68k approved these changes Dec 5, 2025

View reviewed changes

ivoanjo and others added 3 commits December 5, 2025 16:53

Update process-context/c-and-cpp/example_ctx.c

0a0b964

Add pid when printing context in example Co-authored-by: Christos Kalkanis <christos.kalkanis@elastic.co>

Tweak printf to make it clear pid is not part of the context

2e61364

Add pause point after each burn cpu step

ca75c64

ivoanjo added a commit to ivoanjo/opentelemetry-specification that referenced this pull request Dec 8, 2025

Update spec to reflect that named mappings are not always available o…

4c871ac

…n Linux 5.17+ See open-telemetry/sig-profiling#23 for a wider discussion of this.

Reorder struct fields as per upstream spec discussion

6963854

See also open-telemetry/opentelemetry-specification#4719 (comment)

Gandem approved these changes Dec 9, 2025

View reviewed changes

felixge approved these changes Dec 10, 2025

View reviewed changes

felixge merged commit 7fd7125 into open-telemetry:main Dec 10, 2025
1 check passed

christos68k mentioned this pull request Dec 10, 2025

OTEP: Process Context: Sharing Resource Attributes with External Readers open-telemetry/opentelemetry-specification#4719

Merged

5 tasks

ivoanjo mentioned this pull request Dec 18, 2025

Update experimental C/C++ implementation for process context OTEP with memfd + in-place modification #34

Merged

Conversation

ivoanjo commented Dec 1, 2025

Uh oh!

christos68k left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ivoanjo commented Dec 5, 2025

Uh oh!

christos68k commented Dec 5, 2025

Uh oh!

ivoanjo commented Dec 5, 2025

Uh oh!

christos68k commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivoanjo commented Dec 5, 2025

Uh oh!

ivoanjo commented Dec 5, 2025

Uh oh!

ivoanjo commented Dec 5, 2025

Uh oh!

christos68k left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

christos68k Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

ivoanjo Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

felixge left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

christos68k commented Dec 5, 2025 •

edited

Loading