Clarify log collection instructions for Datadog Agent #33203
Conversation
Updated instructions for log collection with Datadog Agent to clarify trace ID injection and automatic detection of OpenTelemetry keys.
Preview links (active after the
|
| - **Trace ID format**: Datadog automatically detects standard OpenTelemetry trace context keys (`trace_id` and `span_id`) in JSON logs. No manual configuration is required. | ||
| - If you are using **Datadog's tracing libraries** (like `dd-trace-py`), this is handled for you automatically. | ||
| - If you are generating logs with OpenTelemetry `trace_id` and `span_id` (as shown in the [file-scraping example](#scrape-logs-from-files)), you must use a [Log Processing Rule][5] in Datadog to remap your `trace_id` attribute to `dd.trace_id`. | ||
|
|
||
| - If you are defining custom attribute names for your trace IDs (not `trace_id` or `span_id`), you must use a [Log Processing Rule][5] to remap your custom attribute to dd.trace_id. |
There was a problem hiding this comment.
A few nitpicks:
- Datadog's tracing libraries don't use the
trace_id/span_idformat, so the second sentence is a bit confusing - Plenty of OTel log bridges don't actually follow the OTel standard unfortunately, so problematic attribute names are not necessarily "custom"
- (In the existing text) As I understand it, the Trace/Span ID Remapper rules don't rename attributes to DD conventions, they just set an internal field
So I would suggest wording like:
Datadog automatically detects the
dd.trace_idanddd.span_idconvention used by Datadog's tracing libraries, as well as the OpenTelemetry standardstrace_idandspan_id.
If your logging instrumentation uses a different attribute name for your trace/span IDs, you must use a Trace/Span ID Remapper processing rule to make sure it is recognized.
It may also be good to add a link to this OTel docs page to justify that trace_id / span_id really is the standard.
Alternatively, instead of suggesting the Trace ID remapper, we could link to this docs page which explains how to configure log preprocessing to recognize the current attributes? It seems simpler than a remapper rule to me, but I'm not sure what the logs team would recommend between the two, they seem otherwise equivalent to me.
There was a problem hiding this comment.
Thank you! Very helpful. I adjusted the content further with your extra details.
I'll add someone from logs as a reviewer as well to confirm the trace ID remapper approach.
Updated instructions for log collection with Datadog Agent to clarify trace ID injection and automatic detection of OpenTelemetry keys.
What does this PR do? What is the motivation?
trace_idfields.Merge instructions
Merge readiness:
For Datadog employees:
Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.
Additional notes