Add integration for Gunicorn

[toumorokoshi]

Starting a discussion around what integration for Gunicorn would look like, and also discuss if it's needed.

Today, we already have integrations with web frameworks directly (e.g. flask) as well as wsgi:

https://github.com/open-telemetry/opentelemetry-python/tree/master/ext

I presume that, if we support the major gateway interfaces (WSGI and ASGI), and finer grained support per framework, then everything should work fine with Gunicorn with support for ASGI in benoitc/gunicorn#1380.

But wanted to get further thoughts.

[Oberon00]

The main thing I can think of that gunicorn could provide us with, is the initial start time of the request in the WSGI environment. Then we can use that when starting the span to include the time it took to dispatch the request to the application.

[disfluxly]

I think this would be highly beneficial as well. It's a very common model to use Gunicorn -> Flask, and it's not entirely uncommon to see errors popping up when Gunicorn attempts to spin up the Flask application factory (especially for multi-process setups). Typically this would occur before the opentelemetry-ext-flask would get initiated, so having this integration would provide some great out-of-the-box insight.

[hacst]

Making gunicorn just work would definitely be great. Things I stumbled over so far:

• Generally gunicorn is run multiprocess so things like prometheus export become quite tricky (see https://github.com/prometheus/client_python#multiprocess-mode-gunicorn). Not sure the SDK in general will behave as expected.
• Gunicorn writes the access log after closing the wsgi request so annotating the log message with traceparent for correlation is tricky

[ocervell]

This would be beneficial indeed. Gunicorn is widely used to run Flask apps. Flask integration won't work with gunicorn since things like request_count would get a separate / process number and we'll have to aggregate across processes. Usually this result in deduplication of timeseries and increased cost.

Custom metrics
A big problem right now is for counters across processes, it seems like the only way is to have 1 separate counter per process, but we would like 1 counter shared across all processes.

Any ideas for workarounds there ? Maybe if we use OT Collector and do the aggregation there we could achieve something like this ?

Framework metrics
gunicorn only supports native instrumentation with statsd by passing the statsd- flags. If we can have OT as a statsd receiver that can forward metrics, we're solving that part. Currently I'm using a statsd-exporter to Prometheus but would like to have a native OT way to do this.

[toumorokoshi]

Framework metrics
gunicorn only supports native instrumentation with statsd by passing the statsd- flags. If we can have OT as a statsd receiver that can forward metrics, we're solving that part. Currently I'm using a statsd-exporter to Prometheus but would like to have a native OT way to do this.

AFAIK metrics is still in a bit of flux, so waiting might be a good choice. Unless @lzchen has some thoughts there. I guess it is possible today to use the API exclusively to record measures, so if you're ok with some potential backwards-incompatible API changes down the road we could add them.

[lzchen]

@ocervell
It would be great if you could bring this up as an issue in the metrics SIG on Thursdays.

[ocervell]

@lzchen i'm in EMEA so the meeting is very late for me. Could you please bring it up ? I think we can find a good implementation of this in the Flask Prometheus exporter which support gunicorn directly. See multiprocess section in their docs.

[aabmass]

I can talk about this in the meeting :)

[lzchen]

@ocervell
No worries. Just an FYI, the meetings are different times alternating weekly, so if you are interested, I believe next week might be a more reasonable time for you.

[jmacd]

Sorry that we didn't get to this topic today. It presents some interesting questions--the idea of aggregating per-process metric events could definitely work, ideally this is something the collector could do. It's also possible to imagine repurposing the OTel-Python Metrics SDK to perform the aggregation itself, I think, but there are some questions this raises that I haven't worked through myself.

[trallnag]

@ocervell The Prometheus Flask Exporter you pointed does not implement multiprocess functionality itself. There are just a few wrappers for convenience. Still using Prometheus client library in the background

[ocervell]

Are there plans to support this in the roadmap yet ?

[srikanthccv]

uWSGI and Gunicorn are widely used for production deployment. It would be great to have them supported.