Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLASK] added request and response hook #416

Merged
merged 42 commits into from
Jun 1, 2021
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
a0cdcc2
added request and response hook - may be very buggy
NickSulistio Apr 6, 2021
53ddfcd
Merge branch 'main' into flask_hooks
NickSulistio Apr 8, 2021
c0ab6a5
added req/resp hooks and tests
NickSulistio Apr 8, 2021
ddb3077
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
NickSulistio Apr 8, 2021
ea965d1
Merge branch 'flask_hooks' of https://github.com/nicksulistio/opentel…
NickSulistio Apr 8, 2021
74bc40c
Merge branch 'main' into flask_hooks
NickSulistio Apr 9, 2021
8a4f2a3
Update CHANGELOG.md
NickSulistio Apr 12, 2021
4721cdf
removed name callback
NickSulistio Apr 13, 2021
79d04fe
Merge branch 'main' into flask_hooks
NickSulistio Apr 13, 2021
f975067
Merge branch 'main' into flask_hooks
lzchen Apr 15, 2021
1605884
Merge branch 'main' into flask_hooks
NickSulistio Apr 16, 2021
7bffdba
added response header tests
NickSulistio May 24, 2021
84bbec1
Merge branch 'main' into flask_hooks
NickSulistio May 25, 2021
5eaeaf2
solved merge conflicts
NickSulistio May 25, 2021
42e9372
fixed more merge conflicts
NickSulistio May 25, 2021
dc63d37
fixed bug where tracer was being unused
NickSulistio May 25, 2021
738065c
Merge branch 'main' into flask_hooks
NickSulistio May 26, 2021
a520dfa
removed venv
NickSulistio May 26, 2021
a308044
Merge branch 'flask_hooks' of https://github.com/nicksulistio/opentel…
NickSulistio May 26, 2021
2757171
added if callable check
NickSulistio May 26, 2021
327905e
Merge branch 'main' into flask_hooks
lzchen May 26, 2021
6dd503c
Update instrumentation/opentelemetry-instrumentation-flask/src/opente…
NickSulistio May 27, 2021
2f2611e
removed useless comment
NickSulistio May 27, 2021
e941831
Merge branch 'main' into flask_hooks
lzchen Jun 1, 2021
34e7013
tried linting
NickSulistio Jun 1, 2021
5534ed1
Merge branch 'main' of https://github.com/open-telemetry/opentelemetr…
NickSulistio Jun 1, 2021
b867ead
linted and merged
NickSulistio Jun 1, 2021
ddc10d7
Merge branch 'flask_hooks' of https://github.com/nicksulistio/opentel…
NickSulistio Jun 1, 2021
cbc0995
linting and merge
NickSulistio Jun 1, 2021
d4eab75
removed venv
NickSulistio Jun 1, 2021
422d67d
removed venv
NickSulistio Jun 1, 2021
dafb554
Merge branch 'flask_hooks' of https://github.com/nicksulistio/opentel…
NickSulistio Jun 1, 2021
d25c675
Revert "linting and merge"
NickSulistio Jun 1, 2021
e91fb0b
Revert "Merge branch 'flask_hooks' of https://github.com/nicksulistio…
NickSulistio Jun 1, 2021
aa37da5
Revert "Revert "Merge branch 'flask_hooks' of https://github.com/nick…
NickSulistio Jun 1, 2021
7b017ec
Revert "Revert "linting and merge""
NickSulistio Jun 1, 2021
add1367
reverted
NickSulistio Jun 1, 2021
a8f353e
Revert "tried linting"
NickSulistio Jun 1, 2021
1aafe2b
Revert "tried linting"
NickSulistio Jun 1, 2021
e95313f
Merge branch 'flask_hooks' of https://github.com/nicksulistio/opentel…
NickSulistio Jun 1, 2021
ff10d2b
reverted to d4eab7503c
NickSulistio Jun 1, 2021
52cfe53
fixed generate
NickSulistio Jun 1, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- `opentelemetry-instrumentation-urllib3` Add urllib3 instrumentation
([#299](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/299))

- `opentelemetry-instrumentation-flask` Added `request_hook` and `response_hook` callbacks.
([#416](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/416))

- `opentelemetry-instrumenation-django` now supports request and response hooks.
([#407](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/407))
- `opentelemetry-instrumentation-falcon` FalconInstrumentor now supports request/response hooks.
Expand All @@ -36,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Remove `http.status_text` from span attributes
([#406](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/406))


## [0.19b0](https://github.com/open-telemetry/opentelemetry-python-contrib/releases/tag/v0.19b0) - 2021-03-26

- Implement context methods for `_InterceptorChannel`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ def hello():
from opentelemetry.util._time import _time_ns
from opentelemetry.util.http import get_excluded_urls


_logger = getLogger(__name__)

_ENVIRON_STARTTIME_KEY = "opentelemetry-flask.starttime_key"
Expand All @@ -78,8 +79,7 @@ def get_default_span_name():
span_name = otel_wsgi.get_default_span_name(flask.request.environ)
return span_name


def _rewrapped_app(wsgi_app):
def _rewrapped_app(wsgi_app, response_hook=None):
def _wrapped_app(wrapped_app_environ, start_response):
# We want to measure the time for route matching, etc.
# In theory, we could start the span here and use
Expand All @@ -101,21 +101,23 @@ def _start_response(status, response_headers, *args, **kwargs):
"missing at _start_response(%s)",
status,
)

if response_hook:
NickSulistio marked this conversation as resolved.
Show resolved Hide resolved
response_hook(span, status, response_headers)

return start_response(status, response_headers, *args, **kwargs)

return wsgi_app(wrapped_app_environ, _start_response)

return _wrapped_app


def _wrapped_before_request(name_callback):
def _wrapped_before_request(request_hook=None):
def _before_request():
if _excluded_urls.url_disabled(flask.request.url):
return

flask_request_environ = flask.request.environ
span_name = name_callback()
span_name = get_default_span_name()
token = context.attach(
extract(flask_request_environ, getter=otel_wsgi.wsgi_getter)
)
Expand All @@ -127,6 +129,9 @@ def _before_request():
kind=trace.SpanKind.SERVER,
start_time=flask_request_environ.get(_ENVIRON_STARTTIME_KEY),
)
if request_hook:
request_hook(span, flask_request_environ)

if span.is_recording():
attributes = otel_wsgi.collect_request_attributes(
flask_request_environ
Expand All @@ -143,7 +148,8 @@ def _before_request():
flask_request_environ[_ENVIRON_ACTIVATION_KEY] = activation
flask_request_environ[_ENVIRON_SPAN_KEY] = span
flask_request_environ[_ENVIRON_TOKEN] = token



return _before_request


Expand All @@ -170,16 +176,15 @@ def _teardown_request(exc):

class _InstrumentedFlask(flask.Flask):

name_callback = get_default_span_name

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)

self._original_wsgi_ = self.wsgi_app
self.wsgi_app = _rewrapped_app(self.wsgi_app)
self.wsgi_app = _rewrapped_app(self.wsgi_app, _InstrumentedFlask.response_hook)

_before_request = _wrapped_before_request(
_InstrumentedFlask.name_callback
_InstrumentedFlask.request_hook
)
self._before_request = _before_request
self.before_request(_before_request)
Expand All @@ -192,25 +197,25 @@ class FlaskInstrumentor(BaseInstrumentor):

See `BaseInstrumentor`
"""

def _instrument(self, **kwargs):
self._original_flask = flask.Flask
name_callback = kwargs.get("name_callback")
if callable(name_callback):
_InstrumentedFlask.name_callback = name_callback
request_hook = kwargs.get("request_hook")
response_hook = kwargs.get("response_hook")
_InstrumentedFlask.request_hook = request_hook
NickSulistio marked this conversation as resolved.
Show resolved Hide resolved
_InstrumentedFlask.response_hook = response_hook
flask.Flask = _InstrumentedFlask

def instrument_app(
self, app, name_callback=get_default_span_name
self, app, request_hook=None, response_hook=None
): # pylint: disable=no-self-use
if not hasattr(app, "_is_instrumented"):
app._is_instrumented = False

if not app._is_instrumented:
app._original_wsgi_app = app.wsgi_app
app.wsgi_app = _rewrapped_app(app.wsgi_app)
app.wsgi_app = _rewrapped_app(app.wsgi_app, response_hook)

_before_request = _wrapped_before_request(name_callback)
_before_request = _wrapped_before_request(request_hook)
app._before_request = _before_request
app.before_request(_before_request)
app.teardown_request(_teardown_request)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -185,46 +185,127 @@ def test_exclude_lists(self):
self.assertEqual(len(span_list), 1)


class TestProgrammaticCustomSpanName(
# class TestProgrammaticCustomSpanName(
NickSulistio marked this conversation as resolved.
Show resolved Hide resolved
# InstrumentationTest, TestBase, WsgiTestBase
# ):
# def setUp(self):
# super().setUp()

# def custom_span_name():
# return "flask-custom-span-name"

# self.app = Flask(__name__)

# FlaskInstrumentor().instrument_app(
# self.app, name_callback=custom_span_name
# )

# self._common_initialization()

# def tearDown(self):
# super().tearDown()
# with self.disable_logging():
# FlaskInstrumentor().uninstrument_app(self.app)

# def test_custom_span_name(self):
# self.client.get("/hello/123")

# span_list = self.memory_exporter.get_finished_spans()
# self.assertEqual(len(span_list), 1)
# self.assertEqual(span_list[0].name, "flask-custom-span-name")



# class TestProgrammaticCustomSpanNameCallbackWithoutApp(
# InstrumentationTest, TestBase, WsgiTestBase
# ):
# def setUp(self):
# super().setUp()

# def custom_span_name():
# return "instrument-without-app"

# FlaskInstrumentor().instrument(name_callback=custom_span_name, request_hook=None)
# # pylint: disable=import-outside-toplevel,reimported,redefined-outer-name
# from flask import Flask

# self.app = Flask(__name__)

# self._common_initialization()

# def tearDown(self):
# super().tearDown()
# with self.disable_logging():
# FlaskInstrumentor().uninstrument()

# def test_custom_span_name(self):
# self.client.get("/hello/123")

# span_list = self.memory_exporter.get_finished_spans()
# self.assertEqual(len(span_list), 1)
# self.assertEqual(span_list[0].name, "instrument-without-app")

class TestProgrammaticHooks(
InstrumentationTest, TestBase, WsgiTestBase
):
def setUp(self):
super().setUp()

def custom_span_name():
return "flask-custom-span-name"
hook_headers = (
"hook_attr",
"hello otel",
)

def request_hook_test(span, environ):
span.update_name("name from hook")

def response_hook_test(span, environ, response_headers):
span.set_attribute("hook_attr", "hello world")
response_headers.append(hook_headers)
NickSulistio marked this conversation as resolved.
Show resolved Hide resolved

self.app = Flask(__name__)

FlaskInstrumentor().instrument_app(
self.app, name_callback=custom_span_name
self.app, request_hook=request_hook_test, response_hook=response_hook_test
)

self._common_initialization()

def tearDown(self):
super().tearDown()
with self.disable_logging():
FlaskInstrumentor().uninstrument_app(self.app)

def test_hooks(self):
expected_attrs = expected_attributes(
{"http.target": "/hello/123", "http.route": "/hello/<int:helloid>", "hook_attr":"hello world"}
)

def test_custom_span_name(self):
self.client.get("/hello/123")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also test adding a header to the response and making sure it was seen here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't mind, could you elaborate on how to do that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd need to update the post response hook and inject headers into the response there. Look at Flask or WSGI docs (which ever object is passed to the response hook). Then you'd capture the result from client.get() call here and verify the result has the same headers you set in the hook earlier.


span_list = self.memory_exporter.get_finished_spans()
self.assertEqual(len(span_list), 1)
self.assertEqual(span_list[0].name, "flask-custom-span-name")

self.assertEqual(span_list[0].name, "name from hook")
self.assertEqual(span_list[0].attributes, expected_attrs)

class TestProgrammaticCustomSpanNameCallbackWithoutApp(
class TestProgrammaticHooksWithoutApp(
InstrumentationTest, TestBase, WsgiTestBase
):
def setUp(self):
super().setUp()

def custom_span_name():
return "instrument-without-app"
hook_headers = (
"hook_attr",
"hello otel without app",
)

def request_hook_test(span, environ):
span.update_name("without app")

FlaskInstrumentor().instrument(name_callback=custom_span_name)
def response_hook_test(span, environ, response_headers):
span.set_attribute("hook_attr", "hello world without app")
response_headers.append(hook_headers)

FlaskInstrumentor().instrument(request_hook=request_hook_test, response_hook=response_hook_test)
# pylint: disable=import-outside-toplevel,reimported,redefined-outer-name
from flask import Flask
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't flask being imported globally above? why do we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Owais, so if I don't import Flask here, no span will be created. I'm not sure exactly why this is the case but when I removed that import line, the span was no longer being created.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'll be able to solve it by creating your flask app instance after calling instrument(). So for each test case, first call .instument() and then create flask app instance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Owais, if I'm not mistaken, aren't I already calling .instrument() before I import Flask and create the app instance?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the app being created anywhere in this method. May be it is being created in the super-class in which case it would be created before since super().setUp() is being called before instrument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey Owais, for this test, we're testing the hooks without the App being created. I'm not entirely sure how this is supposed to work, but previously there was a test for TestProgrammaticCustomSpanNameWithoutApp, which did the same. I believe that @lonewolf3739 may have a better understanding of what this is used for.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NickSulistio Sorry for getting back late. I think If you do import from flask import Flask first and next call FlaskInstrumentor().instrument()then instrumentation doesn't happen because the way patching is done; I remember this was the reason but I may be wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a limitation with the instrumentation itself, I think we can live with it for now and try to improve the instrumentation later.


Expand All @@ -237,9 +318,13 @@ def tearDown(self):
with self.disable_logging():
FlaskInstrumentor().uninstrument()

def test_custom_span_name(self):
def test_no_app_hooks(self):
expected_attrs = expected_attributes(
{"http.target": "/hello/123", "http.route": "/hello/<int:helloid>", "hook_attr":"hello world without app"}
)
self.client.get("/hello/123")

span_list = self.memory_exporter.get_finished_spans()
self.assertEqual(len(span_list), 1)
self.assertEqual(span_list[0].name, "instrument-without-app")
self.assertEqual(span_list[0].name, "without app")
self.assertEqual(span_list[0].attributes, expected_attrs)