Skip to content

[db] Add docs for context info propagation #2236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sincejune wants to merge 24 commits into from
Closed

[db] Add docs for context info propagation #2236

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1529979
[db] Add docs for context info propagation
May 6, 2025
ab6e386
Update docs/database/database-spans.md
sincejune May 7, 2025
08b02a8
Update docs/database/sql-server.md
sincejune May 7, 2025
2c26648
Merge branch 'main' into db-propagate-context-info
May 7, 2025
0ccfda5
feedback
May 7, 2025
a6e65f4
feedback
May 7, 2025
886aa37
feedback
May 7, 2025
f633fa2
feedback
May 7, 2025
5b835f8
Update docs/database/database-spans.md
sincejune May 7, 2025
e021083
Update docs/database/database-spans.md
sincejune May 7, 2025
1bcbf0d
feedback
May 7, 2025
e6803f5
Update docs/database/database-spans.md
sincejune May 8, 2025
f986ee2
add single quotes
May 8, 2025
8794c1d
Add co-author
XSAM May 13, 2025
ec00e3c
feedback
May 19, 2025
fa6cf6d
fix format
May 19, 2025
2c0e01a
update baggage.service.name example
May 19, 2025
430957b
Keep only `service.name` for general databases
XSAM May 21, 2025
7f05f5e
Update sql-server doc
XSAM May 21, 2025
5534272
Adjust font format
XSAM May 21, 2025
9876582
Add an example for sql server
XSAM May 21, 2025
89a4baf
Merge remote-tracking branch 'upstream/main' into db-propagate-contex…
XSAM May 21, 2025
0e7ff53
Fix CI
XSAM May 21, 2025
8effebc
Merge remote-tracking branch 'upstream/main' into db-propagate-contex…
XSAM May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .chloggen/db-propagate-context-info.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Use this changelog template to create an entry for release notes.
#
# If your change doesn't affect end users you should instead start
# your pull request title with [chore] or use the "Skip Changelog" label.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db)
component: db

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add doc for context information propagation

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
# The values here must be integers.
issues: [2162]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
35 changes: 35 additions & 0 deletions docs/database/database-spans.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ linkTitle: Spans
- [Notes and well-known identifiers for `db.system.name`](#notes-and-well-known-identifiers-for-dbsystemname)
- [Sanitization of `db.query.text`](#sanitization-of-dbquerytext)
- [Generating a summary of the query](#generating-a-summary-of-the-query)
- [Propagating context to databases](#propagating-context-to-databases)
- [Recommended attributes](#recommended-attributes)
- [Semantic conventions for specific database technologies](#semantic-conventions-for-specific-database-technologies)

<!-- tocstop -->
Expand Down Expand Up @@ -478,6 +480,39 @@ Semantic conventions for individual database systems or specialized instrumentat
MAY specify a different `db.query.summary` format as long as produced summary remains
relatively short and its cardinality remains low comparing to the `db.query.text`.

## Propagating context to databases
Comment thread
sincejune marked this conversation as resolved.
Outdated

**Status**: [Development][DocumentStatus]

Instrumentations SHOULD propagate the context information to the SQL queries following [sqlcommenter](https://google.github.io/sqlcommenter/spec/).

@zacharycmontoya zacharycmontoya May 7, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to share a learning from my org. We've implemented a sqlcommenter-like approach for our instrumentations and we found it more beneficial to prepend the sql comment rather than append to the query text. This runs counter to the sqlcommenter spec, but we found that most databases truncate the query in views like pg_stat_activity, so prepending the attributes offered a better guarantee that they wouldn't be truncated

@sincejune sincejune May 8, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zacharycmontoya Thank you for pointing this out!

I think it makes sense to prepend the sql comment instead. I'll update this PR accordingly if there are no objections. cc @open-telemetry/semconv-db-approvers @XSAM

@XSAM XSAM May 8, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using prepend conflicts the spec of sql commenter. We might need to change the spec of sql commenter if prepend approach is superior than append approach.

@zacharycmontoya Could you elaborate more about this? How serious and often the truncate will happen to certain databases if we use append.

@nenadnoveljic nenadnoveljic May 12, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres limits query text to 1024 characters by default, but you can increase it.

@nenadnoveljic nenadnoveljic May 12, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prepended comments break pg_hint_plan functionality:

EXPLAIN /*+ IndexScan(t) */  /* a */ SELECT * FROM t WHERE n = 1;
                         QUERY PLAN                          
-------------------------------------------------------------
 Index Scan using i on t  (cost=0.15..36.38 rows=13 width=4)
   Index Cond: (n = 1)
(2 rows)

EXPLAIN /* a */ /*+ IndexScan(t) */  SELECT * FROM t WHERE n = 1;
                           QUERY PLAN                            
-----------------------------------------------------------------
 Bitmap Heap Scan on t  (cost=4.26..14.95 rows=13 width=4)
   Recheck Cond: (n = 1)
   ->  Bitmap Index Scan on i  (cost=0.00..4.25 rows=13 width=0)
         Index Cond: (n = 1)
(4 rows)

@sincejune sincejune May 19, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea is to make it flexible by allowing the implementation of propagation for a specific database to choose between append or prepend.

@sincejune sincejune May 19, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified this PR to incorporate the idea.

@trask trask May 9, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about pushing this as a standard in light of google/sqlcommenter#284

@sincejune sincejune May 12, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The plan is to use sqlcommenter only with attributes that do not change frequently. In the SQL Server documentation, I have added the section below. @trask Does this plan seem good to you?

Instrumentations SHOULD make use of SET CONTEXT_INFO to add high-cardinality information(e.g. traceparent) as adding sql comments to queries changes their unique identifier in SQL Server.

@lmolkova lmolkova May 12, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this PR is still about the context propagation, right? trace-id + span-id are pretty much guaranteed to be unique

@sincejune sincejune May 19, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmolkova The traceparent format is more widely used, so I prefer adopting that. What are your thoughts?


### Recommended attributes
Comment thread
sincejune marked this conversation as resolved.
Outdated

| Attribute | Type | Description | Require level | Stability |
|------------------------|--------|---------------------------------------|-------------------|----------------------------------------------------------------|
| `baggage.service.name` | string | Logical name of the service [1] | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |

@pellared pellared May 7, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If traceparent is present I do not think that service.name is needed. Isn't it the best way to have one or another?

Do we even need to define "require level"? Maybe we should just describe the meaning of each sqlcommenter attribute?

@sincejune sincejune May 7, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If traceparent is present I do not think that service.name is needed. Isn't it the best way to have one or another?

Updated, and I keep the Require level to make it as an annotation as well. What do you think about it?

@pellared pellared May 9, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if we need even the require level. I think these are just different propagation options 😉 But we can keep it for now as I think there are other more important comments to be resolved/addressed.

@nenadnoveljic nenadnoveljic May 12, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If traceparent is present I do not think that service.name is needed.

Even if traceparent is present, service.name is still useful. Exact correlation using trace_id requires both a database sample and the full trace for the same request, but because of sampling, many traces and DB executions are missed. Since the chance of both being captured is low (p_trace * p_db_sample), including service.name helps support approximate correlation when exact matching isn’t possible.

| `traceparent` | string | The trace context of current span [2] | `Recommended` [3] | ![Development](https://img.shields.io/badge/-development-blue) |

**[1] `baggage.service.name`:** MUST be the same for all instances of horizontally scaled services. If the value was not specified, SDKs MUST fall back to `unknown_service:` concatenated with [process.executable.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/), e.g. `unknown_service:bash`. If `process.executable.name` is not available, the value MUST be set to `unknown_service`.
Comment thread
pellared marked this conversation as resolved.
Outdated

**[2] `traceparent`:** MUST be in the [text format](https://www.w3.org/TR/trace-context/#traceparent-header).

**[3] `traceparent`:** `tracparent` have extremely high-cardinality. It's RECOMMENDED to propagate this info if the high-cardinality is safe for the behind databases
Comment thread
sincejune marked this conversation as resolved.
Outdated

**Examples:**

- Query with `baggage.service.name`:

```sql
SELECT * FROM songs /* baggage.service.name=music-player%3Aplay */
```

- Query with `baggage.service.name` and `traceparent`

```sql
SELECT * FROM songs /* baggage.service.name=music-player%3Aplay, traceparent=00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01 */
Comment thread
sincejune marked this conversation as resolved.
Outdated
```

## Semantic conventions for specific database technologies

More specific Semantic Conventions are defined for the following database technologies:
Expand Down
21 changes: 21 additions & 0 deletions docs/database/sql-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ linkTitle: SQL Server
<!-- toc -->

- [Spans](#spans)
- [Propagating context to SQL Server](#propagating-context-to-sql-server)
- [Recommended attributes](#recommended-attributes)
- [Metrics](#metrics)

<!-- tocstop -->
Expand Down Expand Up @@ -152,6 +154,25 @@ and SHOULD be provided **at span creation time** (if provided at all):
<!-- END AUTOGENERATED TEXT -->
<!-- endsemconv -->

### Propagating context to SQL Server
Comment thread
sincejune marked this conversation as resolved.
Outdated

**Status**: [Development][DocumentStatus]

Instrumentations SHOULD propagate the context information to the SQL queries following [sqlcommenter](https://google.github.io/sqlcommenter/spec/).

#### Recommended attributes

| Attribute | Type | Description | Require level | Stability |
|------------------------|--------|---------------------------------------|---------------|----------------------------------------------------------------|
| `baggage.service.name` | string | Logical name of the service [1] | `Required` | ![Development](https://img.shields.io/badge/-development-blue) |
| `traceparent` | string | The trace context of current span [2] | `Recommended` | ![Development](https://img.shields.io/badge/-development-blue) |

**[1] `baggage.service.name`:** MUST be the same for all instances of horizontally scaled services. If the value was not specified, SDKs MUST fall back to `unknown_service:` concatenated with [process.executable.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/), e.g. `unknown_service:bash`. If `process.executable.name` is not available, the value MUST be set to `unknown_service`.

**[2] `traceparent`:** MUST be in the [text format](https://www.w3.org/TR/trace-context/#traceparent-header).

Instrumentations SHOULD make use of [SET CONTEXT_INFO](https://learn.microsoft.com/en-us/sql/t-sql/statements/set-context-info-transact-sql?view=sql-server-ver16) to add high-cardinality information(e.g. `traceparent`) as adding sql comments to queries changes their unique identifier in SQL Server.

@zacharycmontoya zacharycmontoya May 7, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add to the guidance here that instrumentations will need to copy over the transaction (if present) to the new set context_info command. When we didn't do this, we ran into customer issues when there was an active transaction

@XSAM XSAM May 8, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate more about what copy over the transaction is actually doing? Could we just run set context_info before every query?

@zacharycmontoya zacharycmontoya May 9, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the LOC I'm referencing for our C# client : https://github.com/DataDog/dd-trace-dotnet/pull/6233/files#diff-29ead36b0177265c725705ce75f5a441417f4c22e99f7080bc8ef0f486b723a7R145-R146

When creating a new set context_info command that we run before the original query, we simply copy the Transaction from the original DbCommand object to the new one. Otherwise, when executing the set context_info command we encountered a InvalidOperationException

@nenadnoveljic nenadnoveljic May 12, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SET context_info must run on the same physical connection as the SQL statement it’s meant to instrument.

@sincejune sincejune May 19, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated


## Metrics

Microsoft SQL Server client instrumentations SHOULD collect metrics according to the general
Expand Down