Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BatchSpanProcessor accesses deleted resources after tracer destruction #904

Closed
benlandrum opened this issue Jul 15, 2021 · 1 comment · Fixed by #911
Closed

BatchSpanProcessor accesses deleted resources after tracer destruction #904

benlandrum opened this issue Jul 15, 2021 · 1 comment · Fixed by #911
Labels
bug Something isn't working release:required-for-ga To be resolved before GA release
Milestone

Comments

@benlandrum
Copy link
Contributor

Describe your environment
Using a version after v1.0.0-rc2, including recent fixes to scopes and StartSpan.

Steps to reproduce
Destroy a tracer provider connected to a batch span processor while it is processing spans.

What is the expected behavior?
Even on tracer provider destruction, which could happen if a user wants to change the tracing configuration, the batch span processor will emit spans with resources intact.

What is the actual behavior?
Resource attributes are absent from spans emitted by the batch span processor on tracer provider destruction.

Additional context
I've observed the following sequence of events.

  1. BatchSpanProcessor::OnEnd adds the span to its circular buffer. The resource has the expected attributes.
  2. BatchSpanProcessor::OnEnd does not notify the worker thread.
  3. The tracer's destructor gets called, due to intentional destruction of the global tracer provider.
  4. That tracer's resource destructor gets called (due to tracer destruction).
  5. BatchSpanProcessor::DrainQueue gets called.
  6. BatchSpanProcessor::Export consumes the circular buffer. The resource (still at the same address) has no attributes.

Perhaps the destruction order needs to be adjusted, or recordables should not store bare pointers to resources.

See the Slack discussion for a similar problem occurring.

@benlandrum benlandrum added the bug Something isn't working label Jul 15, 2021
@lalitb lalitb added the release:required-for-ga To be resolved before GA release label Jul 16, 2021
@lalitb lalitb added this to the 1.0.0 milestone Jul 16, 2021
@benlandrum
Copy link
Contributor Author

One possibility is to rearrange the members of TracerContext, constructing the resource_ before the processor_ (and commenting on order dependence). This way, the the processor would get destroyed before the resource.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working release:required-for-ga To be resolved before GA release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants