Opentelemetry not working correctly with Reactive Spring-Boot

[frommeyerc]

Describe the bug
The current implementation is not working correctly in a reactive Spring-Boot application. The provided auto-configuration does set up a somewhat working tracing instrumentation, but context propagation doesn't work. This seems to be due to the use of a ThreadLocal where this is not appropriate in a reactive application that is not bound to a single thread per request.

Steps to reproduce
I created a minimal configuration here that shows the issue.

What did you expect to see?
I expected to have a working tracing setup where nested spans carry the same traceId.

What did you see instead?
Every span gets a new traceId.

What version are you using?
The repo uses 1.7.0 but I believe nothing relevant changed meanwhile.

Environment
Compiler: AdoptOpenJDK 11.0.2
OS: Ubuntu 20.04

Additional context
There is an open discussion on this topic: #4471 (reply in thread)

[laurit]

@frommeyerc I believe the issue isn't really due to usage of ThreadLocal but because there is no instrumentation for reactor netty that is used as http server in your test application. When your application makes a http request it adds a traceparent header for context propagation but as far as I can tell on the receiving end nothing reads that header so context doesn't propagate. If you'd modify your application to manually restore context with something like

  @Autowired
  private OpenTelemetry openTelemetry;

  @GetMapping("{no}")
  public Mono<String> get(@PathVariable int no, @RequestHeader HttpHeaders httpHeader) {
    Context extractedContext = openTelemetry.getPropagators().getTextMapPropagator()
            .extract(Context.current(), httpHeader, new TextMapGetter<>() {
              @Override
              public Iterable<String> keys(HttpHeaders carrier) {
                return carrier.keySet();
              }

              @Override
              public String get(HttpHeaders carrier, String key) {
                return carrier.getFirst(key);
              }
            });

    try (Scope scope = extractedContext.makeCurrent()) {
      log.info("Called with {}", no);
      Mono<String> mono = no < 10 ? callDown(no + 1) : Mono.just("Hi");
      return ContextPropagationOperator.runWithContext(mono, Context.current());
    }
  }

it should behave a bit better. I changed opentelemetry version to 1.9.0, ContextPropagationOperator.runWithContext was missing in 1.7.0 that you used.
When using javaagent context propagation seems to work without any extra steps needed.

[frommeyerc]

@laurit this seems to work. Movin the code into a WebFilter this even seems to work as a library. I'll do a couple more checks and if this turns out good, I'll close this one here. Thank you very much for that hint. Would there have been some fine documentation for me to read this up (and to find even more information now)?

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed automatically if there is no response from the author within 7 additional days from this comment.