Requests integration #94

Oberon00 · 2019-08-19T16:10:18Z

This (draft) PR adds an integration for the requests libraray (see README.md).

~~For some reason, pylint insists on an __init__.py in the module root for requests, but not for wsgi. I'm again clueless as to why. 😅~~ I seem to have fixed this by renaming the module.

~~More tests will follow.~~ ✔️

reyang · 2019-08-21T15:00:08Z

ext/opentelemetry-ext-http-requests/src/opentelemetry/ext/http_requests/__init__.py

+            # TODO: The OTel spec says "SpanKind" MUST be "Client" but that
+            #  seems to be a leftover, as Spans have no explicit "Client"
+            #  field.
+            span.set_attribute("http.method", method.upper())


Curious, when do we prefer double quote vs. single quote for literal strings?

Ah, true. I have been inconsistent here. Normally I just use single quotes for everything but I read up on Black and it seems to format everything with double quotes.

reyang · 2019-08-21T15:08:23Z

ext/opentelemetry-ext-http-requests/src/opentelemetry/ext/http_requests/__init__.py

+
+            return result
+
+        # TODO: How to handle exceptions? Should we create events for them? Set


We will need to capture exception, call Span.SetStatus, and rethrow if we got anything.
I guess we don't need to create event for exception as the user might not want it (e.g. the user might decide to capture exception at higher level and attach event). Having a TODO is good to go for now, this can be a separate PR.

reyang · 2019-08-21T15:09:13Z

ext/opentelemetry-ext-http-requests/README.rst

+    import opentelemetry.ext.http_requests
+    from opentelemetry.trace import tracer
+
+    enable_requests_integration.enable(tracer())


Where is enable_requests_integration?

a-feld · 2019-08-22T04:09:53Z

ext/opentelemetry-ext-http-requests/src/opentelemetry/ext/http_requests/__init__.py

+
+    # TODO: We should also instrument requests.sessions.Session.send
+    # but to avoid doubled spans, we would need some context-local
+    # state (i.e., only create a Span if the current context's URL is


Would it work to instrument only requests.sessions.Session.send?

Then we would see each redirect separately but we wouldn't have an overarching span for the complete logical request including redirects.

reyang

I know we have several TODOs, but this is good enough to move forward :)

ext/opentelemetry-ext-http-requests/tests/test_requests_integration.py

reyang

LGTM, we can leave the TODOs for another PR.

a-feld

Agree with @reyang , this seems like a good start for this integration and the limitations / TODOs are clearly spelled out! 🎉

Oberon00 · 2019-08-22T17:38:18Z

I will wait for #95 to be merged because I know that I at least used assert_called or something like that and after rebasing & fixing CI, I will merge.

toumorokoshi · 2019-08-24T03:44:48Z

ext/opentelemetry-ext-http-requests/setup.cfg

+# limitations under the License.
+#
+[metadata]
+name = opentelemetry-ext-http-requests


why "http-requests"? It feels to me like calling this "opentelemetry-ext-requests" might be clearer.

For example, I imagine flask integration being along the lines of "opentelemetry-ext-flask" rather than "opentelemetry-ext-httpserver-requests".

Trying to put how I feel into words, I think one who is looking for an integration will find it via the package name alone, and the additional categorization is unneeded.

@toumorokoshi I kind of agree with you, this is what I did in OpenCensus https://github.com/census-instrumentation/opencensus-python/tree/master/contrib.

I agree but it seems that pylint gets confused when naming it requests. Please look at the commit history of this PR.

toumorokoshi · 2019-08-24T03:46:57Z

ext/opentelemetry-ext-http-requests/src/opentelemetry/ext/http_requests/__init__.py

+# if the SDK/tracer is already using `requests` they may, in theory, bypass our
+# instrumentation when using `import from`, etc. (currently we only instrument
+# a instance method so the probability for that is very low).
+def enable(tracer):


could this be refactored to allow one to invoke injecting the tracer into a single requests Session?

I think the global patch should be a provided method as well, but I think it's important to allow consumers to configure multiple http clients. A global patch doesn't provide that flexibility.

toumorokoshi · 2019-08-24T03:53:49Z

ext/opentelemetry-ext-http-requests/setup.cfg

+packages=find:
+install_requires =
+    opentelemetry-api >= 0.1.dev0
+    requests ~= 2.0


this may cause issues if a consumer wants to use requests greater than 2.0. I would argue we should err on the side of minimal version range restrictions to reduce friction as the requests projects revs their version for whatever reason.

You mean, we should allow requests3? From reading its homepage it seems to be significantly different though.

Perhaps >=2.0,<3.0 ?

I think requests ~= 2.0 is equivalent to >=2.0,<3.0.

From pep 440, I think this translates to >=2.0,==2.*

See previous CI failure for pylint issue: ************* Module ext/opentelemetry-ext-requests/src/__init__.py ext/opentelemetry-ext-requests/src/__init__.py:1:0: F0001: No module named ext/opentelemetry-ext-requests/src/__init__.py (fatal) It seems that pylint gets confused when there is more than one "requests" module??

Co-Authored-By: Allan Feldman <[email protected]>

Oberon00 · 2019-08-27T13:59:12Z

I'm merging this now, but if someone finds a way to change the name from http-requests to requests without breaking pylint, I'm happy for follow-up PRs!

Oberon00 · 2019-08-28T12:37:21Z

Implements #79

Oberon00 force-pushed the requests-integration branch 2 times, most recently from 2f97089 to 45ecfa1 Compare August 21, 2019 14:42

Oberon00 marked this pull request as ready for review August 21, 2019 14:44

Oberon00 requested review from c24t, carlosalberto and reyang as code owners August 21, 2019 14:44

reyang reviewed Aug 21, 2019

View reviewed changes

a-feld reviewed Aug 22, 2019

View reviewed changes

reyang approved these changes Aug 22, 2019

View reviewed changes

a-feld reviewed Aug 22, 2019

View reviewed changes

ext/opentelemetry-ext-http-requests/tests/test_requests_integration.py Outdated Show resolved Hide resolved

Oberon00 requested review from reyang and a-feld August 22, 2019 12:10

reyang approved these changes Aug 22, 2019

View reviewed changes

a-feld approved these changes Aug 22, 2019

View reviewed changes

toumorokoshi reviewed Aug 24, 2019

View reviewed changes

Oberon00 and others added 10 commits August 26, 2019 18:21

Add requests integration.

eb972ea

Fix README code sample.

00e72c8

request test: disable the patcher.

171e544

requests test: Use actual Response object.

c66af2f

Co-Authored-By: Allan Feldman <[email protected]>

requests: Consistently use double quotes.

652782f

requests: Fix REAMDE.

6767d0b

requests: Add TODO for propagation.

a836cc9

requests test: Properly clean up after ourselves.

9fc87b4

requests: Add test for unparseable URL.

6a98f52

Oberon00 force-pushed the requests-integration branch from 5b56ec3 to 6a98f52 Compare August 26, 2019 16:24

Oberon00 requested a review from lzchen as a code owner August 26, 2019 16:24

Oberon00 added 5 commits August 26, 2019 18:28

Fix envlist.

f8e2f4d

Fix test for py 3.5

f4258c7

Fix tests.

ab99504

Fix tests 2.

a2df1ac

Fix urllib3 ValueError on Py 3.4.

8fa2090

Oberon00 merged commit b868b5a into open-telemetry:master Aug 27, 2019

Oberon00 deleted the requests-integration branch August 28, 2019 12:37

a-feld mentioned this pull request Aug 29, 2019

REQUEST: New membership for @a-feld open-telemetry/community#165

Closed

6 tasks

Oberon00 mentioned this pull request Jan 24, 2020

Change imports to avoid pylint namespace errors #376

Merged

srikanthccv pushed a commit to srikanthccv/opentelemetry-python that referenced this pull request Nov 1, 2020

Add Logger interface (open-telemetry#94)

14310c3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Requests integration #94

Requests integration #94

Oberon00 commented Aug 19, 2019 •

edited

Loading

reyang Aug 21, 2019

Oberon00 Aug 22, 2019

reyang Aug 21, 2019

reyang Aug 21, 2019

a-feld Aug 22, 2019

Oberon00 Aug 22, 2019

reyang left a comment

reyang left a comment

a-feld left a comment

Oberon00 commented Aug 22, 2019

toumorokoshi Aug 24, 2019

reyang Aug 24, 2019

Oberon00 Aug 24, 2019

toumorokoshi Aug 24, 2019

toumorokoshi Aug 24, 2019

Oberon00 Aug 24, 2019

carlosalberto Aug 26, 2019

a-feld Aug 26, 2019

Oberon00 commented Aug 27, 2019

Oberon00 commented Aug 28, 2019


		return result

		# TODO: How to handle exceptions? Should we create events for them? Set

Requests integration #94

Requests integration #94

Conversation

Oberon00 commented Aug 19, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reyang left a comment

Choose a reason for hiding this comment

reyang left a comment

Choose a reason for hiding this comment

a-feld left a comment

Choose a reason for hiding this comment

Oberon00 commented Aug 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Oberon00 commented Aug 27, 2019

Oberon00 commented Aug 28, 2019

Oberon00 commented Aug 19, 2019 •

edited

Loading