Add initial tracer API

[c24t]

This PR adds the tracing API package and proposes some conventions for the package structure, documentation, and style of the project. As it is now, the API only describes the required classes for using the tracer to record spans. Propagation formats, resources, and the context API are all out of scope for this PR.

The API here borrows from:

• The current opentelemetry-java API (generate the javadocs and see .../docs/io/opentelemetry/trace/Tracer.html)
• The WIP OpenTelemetry tracing spec
• The OpenTracing spec
• The OpenCensus tracing spec
• The existing opencensus-python tracing implementation

The OT java API is our current best source of truth. Ideally we can use this PR to prove the java API and push the discussion here up to the spec.

[c24t]

These docs are WIP, I just wanted a good example module docstring here to test the generated html docs.

[reyang]

How would the implicit one and explicit one work together in a mixed mode?
For example, I have a function A which is using implicit in-proc propagation, another function B which is using explicit propagation, if I need to call A from B, what should I do?

[Oberon00]

I think functions that want to support both styles (e.g. in libraries) will need to add an optional parameter for the parent Span that defaults to the current one.

[reyang]

Going down this path, I would imagine either all the libraries end up having parent span as the optional parameter for all the exposed functions, or most libraries will only support implicit context and makes it super difficult for the application developer to use explicit context.

[c24t]

Supporting both implicit and explicit context like this is going to make for an ugly implementation. The spec should probably describe what happens for each, and I think there's good reason to separate the span constructors for each. We may need to solve this in the spec.

[carlosalberto]

So honestly adding support for explicit parent at the core API shouldn't look that bad - for libraries (i.e. Django instrumentation), I think it's fine to make it super difficult for the user to override it, or even do not expose it at all IHMO.

[reyang]

How about libraries? The application could use explicit propagation consistently, while calling into a library which expects implicit context propagation.

[Oberon00]

Yeah, we definitely should think about and document best practices for what libs should do. For that question, see https://github.com/open-telemetry/opentelemetry-python/pull/11/files#r290703075. Another question is what tracer they should use (e.g.: Use global tracer? Pass tracer at every call? Provide way to set library-wide tracer? Provide way to set a library-wide callback function that it calls to get the tracer?)

[carlosalberto]

Best-practices aside, users should have the final thing to say - if they, for some reason, need to 'mix' these modes (as you guys call it), that should be supported IHMO.

(As for the other questions, I'd say using a global Tracer would be a good thing to have)

[c24t]

The application could use explicit propagation consistently, while calling into a library which expects implicit context propagation.

This is a problem since it would mean that passing an explicit context should change the implicit one. This would be unexpected in the application, but required if it's calling into a library that uses implicit context.

Provide way to set a library-wide callback function that it calls to get the tracer?

I thought implementations would provide a global tracer, do you want the callback so integrations can change this?

[reyang]

Do we only check if name is null or we expect more validations?

[carlosalberto]

I'd personally advocate for using a generic name here, instead of failing.

[c24t]

OK, it sounds like implementations should provide a safe default here. In general, do you expect implementations to be more or less permissive than the API? I.e. should implementations be required to throw errors we list in the API? Should they be allowed to throw errors we don't list?

[reyang]

In general, do you expect implementations to be more or less permissive than the API? I.e. should implementations be required to throw errors we list in the API?

Yes, I bet almost all implementations will have more restriction than the API would allow.

Should they be allowed to throw errors we don't list?

Probably not. One design principle we may want to follow is "the monitoring SDK will try to keep silent instead of changing the application behavior". In this spirit I guess we will have exporters doing some downcast/trimming based on their specific need, and not throw exceptions.

[c24t]

In that case: do we want to allow implementations to throw here on null/blank name by including it in the API or are we ok with forbidding them from throwing by removing it?

[c24t]

I bet almost all implementations will have more restriction than the API would allow

This suggests that implementations will throw errors we don't list in the API, or am I missing something?

FWIW the java API says to raise a NPE here. It might be worth taking this conversation there (or to the spec repo).

Using a safe default span name sounds good to me as long as we get the same behavior in all clients, or decide that this is "unspecified behavior" and that different clients can do this differently.

[carlosalberto]

Personally I'm in favor of option 1.

I'm also in favor one. That being said, this can also be mentioned/discussed in the Specification ;)

[Oberon00]

I'd also prefer option 1. Option 3 (generating a random span name) is no good IMHO, because span names should be low-cardinality.

[c24t]

I added open-telemetry/opentelemetry-specification#141 to track this in specs.

[c24t]

It sounds like we should list any error implementations could throw in the api method docstring. Safe default names sound good to me, but let's clarify this in the spec first. I added #18 to track this.

[reyang]

How to create a root span which has no parent?

[Oberon00]

True. In fact, I think its strange that create_span(..., parent=None) should have a non-None parent. One suggestion: create_span should just never affect or use (as parent) the active span. But instead, enhance the context manager returned by span to return the created span at __enter__.

[c24t]

I can think of a few options here: use a sentinel default value instead of null and take parent=None to mean that we should create a root span, add a separate method create_root_span, or change the way we create spans and match java's builder pattern more closely.

create_span should just never affect or use (as parent) the active span

That's pretty surprising, I'd expect creating a child of the current active span to be the most common use case.

But instead, enhance the context manager returned by span to return the created span at __enter__

That's what I'd expect the context manager to do now.

[Oberon00]

After our meeting today, I'm thinking that maybe we should start out with only explicit context propagation/parent span specification. The implicit propagation (which requires thinking about a definition of an active trace context or span per thread/coroutine/"scope"/??? and whether the tracer should be responsible for it or some separate component that is one layer below or next to the tracer) is probably one of the most difficult APIs to get right.

[reyang]

Maybe I've missed something, I was thinking a different approach:

Implicit

with tracer.span(name):
    do_work()

Explicit

span = tracer.span(name)
span.start()
do_work()
span.finish()

And in the span class we will have __enter__ and __exit__ (or __aenter__ and __aexit__).

[c24t]

That's a nice interface, but what do you do about the fact that we need to modify the context if span is used as a context manager and not otherwise?

[reyang]

For languages which don't have the with or using concept, it might be good to have a dedicated API. For Python, with span (Span.__enter__ and Span.__exit__) could be a good replacement for use_span?

[reyang]

FYI @c24t

https://github.com/census-instrumentation/opencensus-python/blob/master/context/opencensus-context/examples/async_span.py

https://github.com/census-instrumentation/opencensus-python/blob/master/context/opencensus-context/examples/span.py

[c24t]

I agree if we can find a nice way to handle modifying the context.

[c24t]

After our meeting today, I'm thinking that maybe we should start out with only explicit context propagation/parent span specification.

One argument for biting the bullet and figuring out implicit propagation now is that it will affect the API for explicit propagation too. Another is that even in the "explicit propagation" case I'm assuming we don't handle a context object in this API; we just make sure not to touch the implicit context.

Another is that hashing out the tracing/context interaction is one of the reasons for this PR. :D

The implicit propagation (which requires thinking about a definition of an active trace context or span per thread/coroutine/"scope"/??? and whether the tracer should be responsible for it or some separate component that is one layer below or next to the tracer) is probably one of the most difficult APIs to get right.

My working assumption is that our context will be a ContextVar or a thin wrapper around it like @reyang's context package in OC. PEP 555 describes these as context-local (as compared to thread-local) in that they're scope is a single call chain that might include async calls. One reason we need explicit context is to work with non-standard execution models like tornado's, and there's reason to believe that frameworks will start to standardize on contextvars now that the language provides one obvious way to do it. Tornado recently switched to asyncio and deprecated their own context module for this reason.

[reyang]

Per our discussion during the meeting, let's get this merged first. We can make further improves on namespace, package layout, improve span creation and global tracer registration.