Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenTelemetry support #2435

Closed
17 of 18 tasks
pquentin opened this issue Feb 12, 2024 · 1 comment
Closed
17 of 18 tasks

Add OpenTelemetry support #2435

pquentin opened this issue Feb 12, 2024 · 1 comment

Comments

@pquentin
Copy link
Member

pquentin commented Feb 12, 2024

There are currently three ways to instrument an application that uses the Elasticsearch Python client:

The main benefit of the existing approaches is that while they can handle past versions of this client, they're fragile to changes in elasticsearch-py, are difficult to test and may be suboptimal as the code may need refactorings to emit spans with complete information. In the case of the OpenTelemetry instrumentations, they don't have clear owners. For this reason, it makes sense to implement this in elasticsearch-py where the feature will be tested and maintained.


We will follow the existing Semantic conventions for Elasticsearch and emit spans for all requests when the opentelemetry-api optional package will be detected. Correctly configuring OpenTelemetry SDK will be left to the user.

The elasticsearch-ruby OpenTelemetry instrumentation will be used as inspiration:

Configuration

  • OTEL_PYTHON_INSTRUMENTATION_ELASTICSEARCH_ENABLED (default: true) Enable / Disable the OpenTelemetry instrumentation. With this configuration option you can enable (default) or disable the built-in OpenTelemetry instrumentation.
  • OTEL_PYTHON_INSTRUMENTATION_ELASTICSEARCH_CAPTURE_SEARCH_QUERY (default: omit) Capture search request bodies. Per default, the built-in OpenTelemetry instrumentation does not capture request bodies due to data privacy considerations. You can use this option to enable capturing of search queries from the request bodies of Elasticsearch search requests in case you wish to gather this information regardless. The options are to capture the raw search query, sanitize the query with a default list of sensitive keys, or not capture it at all. Valid Options: omit, sanitize, raw
  • OTEL_PYTHON_INSTRUMENTATION_ELASTICSEARCH_SEARCH_QUERY_SANITIZE_KEYS (default: None) Sanitize the Elasticsearch search request body You can configure the list of keys whose values are redacted when the search query is captured. Values must be comma-separated.

Testing

We will have an "OpenTelemetry" test mode that will run the whole test suite with OpenTelemetry enabled and will test a simple API to make sure the exported spans are correct.

The InMemorySpanExporter allows end-to-end testing.

Endpoint id and path parts

Those will need to be passed to the transport.

Steps

Supported attributes

Required

  • db.system
  • db.elasticsearch.path_parts
  • db.operation
  • http.request.method
  • url.full

Recommended

  • db.elasticsearch.cluster.name
  • db.elasticsearch.node.name
  • db.statement
  • server.address
  • server.port
@pquentin
Copy link
Member Author

pquentin commented Jul 2, 2024

Closing as sanitization isn't mandatory for actual support. We can always add it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant