Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capture of http.route to DjangoInstrumentor middleware #1226

Conversation

stschenk
Copy link
Contributor

@stschenk stschenk commented Oct 8, 2020

Description DjangoInstrumentor middleware so that it captures the resolved function path for the route as the span attribute http.route. This will allow the function path to be shown used as an organizing operation name in AppInsights.

Extends

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • [ X ] Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Extended instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py to cover this change

  • [ X ] Test instrumentation/opentelemetry-instrumentation-django/tests/test_middleware.py

Checklist:

  • [ x ] Followed the style guidelines of this project
  • [ x ] Changelogs have been updated
  • [ x ] Unit tests have been added
  • Documentation has been updated

@stschenk stschenk requested a review from a team October 8, 2020 22:30
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Oct 8, 2020

CLA Check
The committers are authorized under a signed CLA.

@@ -151,6 +169,7 @@ def process_response(self, request, response):
self._environ_activation_key in request.META.keys()
and self._environ_span_key in request.META.keys()
):
self._set_http_route(request.META[self._environ_span_key], request)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would doing this in process_request be enough and take care of both success/error cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only been able to find this route as being available from request.resolver_match.route and request.resolver_match is none during process_request() but become available during process_response and process_exception. So I think this value cannot be set just in process_request().

else:
# A function-based view
func_path = func.__module__ + "." + func.__name__
span.set_attribute("http.route", func_path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http.route is supposed to be a route (usually templated or named) from an HTTP router. In django's case, that would be the string representation of a URL from urls.py. Looks like we are using the Python function/handler name. I'm not sure how useful this tag is but irrespective of whether we add it or not, it shouldn't be named http.route as it means something else entirely unless I'm mistaken.
Spec: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/semantic_conventions/http.md#http-server-semantic-conventions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we want is the http.route to be in the span attribute. But you are correct, the value should be populated from the request path (which will be a string from urls.py). I believe it should be within the request parameter.

Copy link
Contributor

@lzchen lzchen Oct 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@stschenk stschenk Oct 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@owais / @lzchen Thank you for your comments.

I chose to use the class/function name that the request is being routed to so that a developer would be able to go directly from the http.route as reported in AppInsights( and other tools) to the code. But I do see that this is not in accordance with the spec so I will make the change so that http.route is set to the "matched route (path template)".

However, I do not agree that the "matched route (path template)" is the request.path as opencensus uses. This is because the request.path is the actual path of the request, not the route or path template. One path template, you could see multiple paths because the parameters in the path will vary. For instance, the following paths from request.path:
/api/1/financial-institutions/111/plaid_details/
/api/1/financial-institutions/444/plaid_details/
/api/1/financial-institutions/554/plaid_details/
would all map to one route:
^api/1/financial-institutions/(?P[^/.]+)/plaid_details/$'

I have only been able to find this route as being available from request.resolver_match.route and request.resolver_match is none during process_request() but become available during process_response and process_exception. So I think this value cannot be set just in process_request().

I will go ahead and make the change to setting http.route to request.resolver_match.route and push up the change.

Please let me know what you think. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, I do not agree that the "matched route (path template)" is the request.path as opencensus uses.

If you are trying to add this to enable your AppInsights scenario, I suggest doing it the same way OpenCensus is doing it. OpenCensus Python SDK is Microsoft's current solution for Python SDK to send telemetry to App Insights and so being consistent with it is important for back compat and to not break users. If you want to propose a new design, I think submitting a separate feature request (after merging this PR) would be good.

Copy link
Contributor Author

@stschenk stschenk Oct 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lzchen - I have change the code so that it uses request.path for the http.route value. I think this make the middleware equivalent to Opencensus.

As you suggested, I will submit a separate feature request around using the route template instead of the url path

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use templated path/route for this and stick with OpenTelemetry spec. If we decide to have same behavior of OC, we'll have same information duplicated in http.target and http.route attributes. I don't think (may be I'm wrong) the breakage here is big enough to warrant going against Otel semantic conventions. OpenCensus users can still find their old value for http.route in OpenTelemetry's http.target so no data is lost after the migration, only renamed. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@owais
I thought request.path represented the templated path/route, but if it isn't then that is my misunderstanding. But yes, I agree with what you are saying.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad we are arriving at a consensus. I have switched the code back to using request.resolver_match.route which is the templated route. I also realized from looking at the OpenCensus implementation that though resolver_match is not set for process_request, it is available in process_view so I added that method.

Also http.path was being set in the OpenCensus implementation by not in the OpenTelemetry implementation so I also added that.

@lzchen
Copy link
Contributor

lzchen commented Oct 9, 2020

Address #1213

span = request.META[self._environ_span_key]

if span.is_recording():
if getattr(request, "resolver_match") and getattr(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: hasattr might be better for this if we aren't using the return val directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used getattr because I also don't want to capture the value if it is None.

Copy link
Contributor

@owais owais Oct 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match = getattr(request, "resolved_match")
if match:
    route = getattr(match, "route")
    if route:
        set_attribute()

Perhaps use this? I know it's usually not a big deal but we never know where the library will be used and better to reduce attribute look ups if we can in case the code ends up on a very hot path.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense... I have not thought about reducing attribute lookups before. Thanks!

@@ -121,6 +122,7 @@ def test_traced_post(self):
self.assertEqual(
span.attributes["http.url"], "http://testserver/traced/"
)
self.assertEqual(span.attributes["http.route"], "^traced/")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to add a templated route so tests cover that and catch any regressions in future.

@@ -117,6 +117,7 @@ def process_request(self, request):
)
for key, value in attributes.items():
span.set_attribute(key, value)
span.set_attribute("http.path", request.path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sound good.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't http.target already being populated in collect_request_attributes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point @lzchen! I will remove the setting of the attribute from the django middleware.

Copy link
Contributor

@owais owais left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment about using http.target instead of http.path

@lzchen lzchen merged commit b54540e into open-telemetry:master Oct 15, 2020
@stschenk stschenk deleted the add_http.route_capture_to_DjangoInstrumentor branch October 15, 2020 20:23
srikanthccv pushed a commit to srikanthccv/opentelemetry-python that referenced this pull request Nov 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants