-
Notifications
You must be signed in to change notification settings - Fork 870
Update OpenTelemetry gen-ai conventions through v1.41 #7497
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jeffhandley
merged 16 commits into
dotnet:main
from
jeffhandley:jeffhandley/otel-genai-1.41.0
May 1, 2026
Merged
Changes from 6 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
176e67d
Add update-otel-genai-conventions skill
jeffhandley ec3c81b
Align OpenTelemetry GenAI instrumentation to v1.40
jeffhandley c1a9949
Emit OpenAI response telemetry attributes
jeffhandley 30e8197
Update GenAI telemetry for v1.41 conventions
jeffhandley 51328d7
Check existing PRs before OTel convention audits
jeffhandley f0ca25b
Document PR metadata for OTel convention updates
jeffhandley 20ec72e
Skip per-chunk delta math when only tracing is enabled
jeffhandley 878deb7
Restructure update-otel-genai-conventions skill per review feedback
jeffhandley 21c6447
Address stylistic PR feedback
jeffhandley d92ebe0
Dedup per-chunk service_tier and system_fingerprint tag writes
jeffhandley 0120286
Consolidate CreateOtelToolDefinition across chat and realtime clients
jeffhandley 7ddadfa
Consolidate identical OTel serialization POCOs into Common/OtelMessag…
jeffhandley b0574d3
Consolidate OTel serialization helpers and operation-error recording …
jeffhandley f1432e3
Extract gen-ai histogram creation to a shared helper
jeffhandley 3eb0e74
Always log gen_ai.client.operation.exception in OpenTelemetryEmbeddin…
jeffhandley 85eed00
Merge branch 'main' into jeffhandley/otel-genai-1.41.0
jeffhandley File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
86 changes: 86 additions & 0 deletions
86
.github/skills/update-otel-genai-conventions/references/change-classification.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| # Change Classification | ||
|
|
||
| Taxonomy for classifying gen-ai changes from semantic-conventions releases. Use this to assess each change's impact on dotnet/extensions. | ||
|
|
||
| ## Classification Categories | ||
|
|
||
| ### 🟢 No Action Required | ||
|
|
||
| | Type | Description | Example | | ||
| |------|-------------|---------| | ||
| | **N/A — No client exists** | Change affects a capability we don't implement (e.g. `retrieval`, `memory`) | `gen_ai.retrieval.*` attributes | | ||
| | **Already implemented** | Change was already implemented in a prior PR | A change that was part of an earlier draft spec we adopted | | ||
| | **Server-side only** | Change affects server/provider-side instrumentation, not client-side | Server span attributes | | ||
| | **Documentation only** | Clarification of existing semantics with no behavioral change | Rewording of attribute descriptions | | ||
|
|
||
| ### 🟡 Minor Action Required | ||
|
|
||
| | Type | Description | Action | | ||
| |------|-------------|--------| | ||
| | **New constant (not emitted)** | New attribute defined but optional or not yet applicable | Add constant to `OpenTelemetryConsts.cs`, skip emission | | ||
| | **Version bump** | Convention version number changed | Update `v1.XX` in doc comments across all OpenTelemetry* files | | ||
| | **Stability promotion** | Attribute moved from experimental to stable | Usually no code change; note in audit table | | ||
|
|
||
| ### 🔴 Code Change Required | ||
|
|
||
| | Type | Description | Action | | ||
| |------|-------------|--------| | ||
| | **New required attribute** | New attribute that should be emitted | Add constant, add emission code, add test assertion | | ||
| | **New metric** | New metric instrument defined | Add metric definition, emission, test | | ||
| | **Attribute rename** | Existing attribute renamed | Update constant value, verify backward compatibility | | ||
| | **New event** | New log/span event defined | Add event via `ILogger` + `[LoggerMessage]`, add test | | ||
| | **Behavioral change** | Change in how existing attributes are populated | Modify emission logic, update test expectations | | ||
| | **New operation name** | New `gen_ai.operation.name` value | Add detection logic, tests | | ||
| | **Schema change** | Change to JSON schema for serialized content (e.g. tool definitions) | Update serialization classes, `[JsonSerializable]` registration | | ||
|
|
||
| ## Indicator Mapping | ||
|
|
||
| Use these indicators consistently in audit reports, implementation summaries, and PR descriptions: | ||
|
|
||
| | Indicator | Category | Meaning | | ||
| |---|---|---| | ||
| | 🟢 | No action required | No compensating code change is needed; explain why. | | ||
| | 🟡 | Minor action required | Small metadata, constant-only, stability, or version-reference update. | | ||
| | 🔴 | Code change required | Runtime behavior, emission logic, metrics, events, serialization, or tests must change. | | ||
|
|
||
| ## Impact Assessment Heuristic | ||
|
|
||
| For each gen-ai change in a release: | ||
|
|
||
| 1. **Does it affect a capability we instrument?** Check the [file inventory](file-inventory.md) for matching client types. | ||
| - No → classify as "N/A — No client exists" | ||
| 2. **Is it already implemented?** Search `OpenTelemetryConsts.cs` for the attribute name. | ||
| - Yes → classify as "Already implemented" | ||
| 3. **Is it client-side or server-side?** Check the semantic convention's `span_kind` or context. | ||
| - Server-side only → classify as "Server-side only" | ||
| 4. **What kind of change is it?** Match to the categories above. | ||
| 5. **How many files need modification?** Count affected files from the file inventory. | ||
| - 1–2 files → Low complexity | ||
| - 3–5 files → Medium complexity | ||
| - 6+ files → High complexity (likely involves shared code or cross-cutting concern) | ||
|
|
||
| ## Audit Table Format | ||
|
|
||
| When presenting the analysis, use this table format: | ||
|
|
||
| ```markdown | ||
| | Semantic Convention Change | Upstream PR | Classification | Action Required | Complexity | | ||
| |---|---|---|---|---| | ||
| | `gen_ai.new.attribute` | [#1234](link) | New required attribute | Add constant + emission + test | Low | | ||
| | `retrieval` operation | [#5678](link) | N/A — No client | None | — | | ||
| | Version reference | — | Version bump | Update doc comments | Low | | ||
| ``` | ||
|
|
||
| ## PR Description Table Format | ||
|
|
||
| When preparing a PR description, adapt the audit table into a concise reviewer-facing table grouped or sorted by semantic-conventions version. Include every analyzed gen-ai change, not just changes that required code edits. | ||
|
|
||
| ```markdown | ||
| | Version | Indicator | Semantic-conventions change | Classification | Compensating change / rationale | | ||
| |---|:---:|---|---|---| | ||
| | v1.XX | 🔴 | `gen_ai.new.attribute` added | New required attribute | Added constant, emission, and tests in `{files}`. | | ||
| | v1.XX | 🟡 | Version reference update | Version bump | Updated OpenTelemetry* doc comments to v1.XX. | | ||
| | v1.XX | 🟢 | Provider server span clarified | Server-side only | No client-side instrumentation change needed. | | ||
| ``` | ||
|
|
||
| The final column should either describe the compensating change made or explain why no code change was made, such as "already implemented", "no local source exists", "no client exists", "server-side only", or "documentation-only clarification". |
73 changes: 73 additions & 0 deletions
73
.github/skills/update-otel-genai-conventions/references/file-inventory.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| # File Inventory | ||
|
|
||
| Files that are typically inspected and/or modified when updating OpenTelemetry gen-ai semantic conventions. | ||
|
|
||
| ## Constants | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs` | All OTel attribute and metric name constants. New attributes/metrics always get constants added here first. | | ||
|
|
||
| ## Instrumentation Clients | ||
|
|
||
| These files contain the actual telemetry emission logic. Each wraps a different AI capability with OTel spans, metrics, and logs. | ||
|
|
||
| | File | Capability | Key Sections | | ||
| |------|-----------|--------------| | ||
| | `src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryChatClient.cs` | Chat completion | Activity creation, attribute emission, message serialization, streaming support, metrics | | ||
| | `src/Libraries/Microsoft.Extensions.AI/ChatCompletion/OpenTelemetryImageGenerator.cs` | Image generation | Activity creation, attribute emission | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Embeddings/OpenTelemetryEmbeddingGenerator.cs` | Embeddings | Activity creation, attribute emission | | ||
| | `src/Libraries/Microsoft.Extensions.AI/SpeechToText/OpenTelemetrySpeechToTextClient.cs` | Speech-to-text | Activity creation, attribute emission | | ||
| | `src/Libraries/Microsoft.Extensions.AI/TextToSpeech/OpenTelemetryTextToSpeechClient.cs` | Text-to-speech | Activity creation, attribute emission | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Realtime/OpenTelemetryRealtimeClientSession.cs` | Realtime sessions | Activity creation, attribute emission | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Realtime/OpenTelemetryRealtimeClient.cs` | Realtime client wrapper | Delegates to session | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Files/OpenTelemetryHostedFileClient.cs` | Hosted file management | Activity creation, attribute emission | | ||
|
|
||
| ## Function Invocation / Tool Orchestration | ||
|
|
||
| These files handle `execute_tool`, `invoke_agent`, and `invoke_workflow` spans: | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `src/Libraries/Microsoft.Extensions.AI/ChatCompletion/FunctionInvokingChatClient.cs` | Chat-based tool orchestration | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Realtime/FunctionInvokingRealtimeClientSession.cs` | Realtime tool orchestration | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Common/FunctionInvocationProcessor.cs` | Shared function invocation logic (used by both chat and realtime) | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Common/FunctionInvocationHelpers.cs` | Shared function invocation helpers | | ||
| | `src/Libraries/Microsoft.Extensions.AI/Common/FunctionInvocationLogger.cs` | Shared function invocation logging | | ||
|
|
||
| ## Shared Code | ||
|
|
||
| | File | Purpose | | ||
| |------|---------| | ||
| | `src/Libraries/Microsoft.Extensions.AI/Common/OpenTelemetryLog.cs` | Shared `[LoggerMessage]` definitions for OTel events (e.g. exception recording) | | ||
| | `src/Libraries/Microsoft.Extensions.AI/TelemetryHelpers.cs` | Shared telemetry helper methods (at library root, not Common/) | | ||
|
|
||
| ## Tests | ||
|
|
||
| | File | Tests For | | ||
| |------|----------| | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/ChatCompletion/OpenTelemetryChatClientTests.cs` | Chat client telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/Image/OpenTelemetryImageGeneratorTests.cs` | Image generator telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/Embeddings/OpenTelemetryEmbeddingGeneratorTests.cs` | Embedding generator telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/SpeechToText/OpenTelemetrySpeechToTextClientTests.cs` | Speech-to-text telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/TextToSpeech/OpenTelemetryTextToSpeechClientTests.cs` | Text-to-speech telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/Realtime/OpenTelemetryRealtimeClientTests.cs` | Realtime session telemetry | | ||
| | `test/Libraries/Microsoft.Extensions.AI.Tests/Files/OpenTelemetryHostedFileClientTests.cs` | Hosted file client telemetry | | ||
|
|
||
| To discover any additional test files: `dir test\Libraries\Microsoft.Extensions.AI.Tests\ -Recurse -Filter "OpenTelemetry*Tests.cs"` | ||
|
|
||
| ## Version References | ||
|
|
||
| The semantic conventions version is referenced in a doc comment in specific OpenTelemetry* instrumentation client files. When bumping the version, update all files that match the grep below — not all OpenTelemetry* files contain the version reference. | ||
|
|
||
| The reference looks like: | ||
|
|
||
| ```csharp | ||
| /// This class provides an implementation of the Semantic Conventions for Generative AI systems v1.XX, | ||
| /// defined at <see href="https://opentelemetry.io/docs/specs/semconv/gen-ai/" />. | ||
| ``` | ||
|
|
||
| Find all occurrences with: | ||
| ``` | ||
| grep -rn "Semantic Conventions for Generative AI systems v" src/Libraries/Microsoft.Extensions.AI/ | ||
| ``` |
67 changes: 67 additions & 0 deletions
67
.github/skills/update-otel-genai-conventions/references/historical-releases.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # Historical Releases | ||
|
|
||
| Mapping of OpenTelemetry semantic-conventions releases with gen-ai changes to dotnet/extensions PRs. | ||
|
|
||
| > **Note**: This file is a point-in-time reference and is not intended to be kept up to date with every new release. It provides context for how past convention updates were handled. For the latest release history, consult the [semantic-conventions releases page](https://github.com/open-telemetry/semantic-conventions/releases) and search the dotnet/extensions PR history. | ||
|
|
||
| ## Release History | ||
|
|
||
| ### Pre-v1.31 (Pre-release Era) | ||
|
|
||
| | Convention Version | dotnet/extensions PR | Description | | ||
| |-------------------|---------------------|-------------| | ||
| | Initial implementation | [#5532](https://github.com/dotnet/extensions/pull/5532) | Initial OpenTelemetry gen-ai instrumentation | | ||
| | v1.29 draft | [#5712](https://github.com/dotnet/extensions/pull/5712) | Align with v1.29 draft conventions | | ||
| | v1.30 | [#5815](https://github.com/dotnet/extensions/pull/5815) | Update to v1.30 conventions | | ||
|
|
||
| ### v1.31–v1.40 (Stable Release Era) | ||
|
|
||
| | Convention Version | dotnet/extensions PR | Description | | ||
| |-------------------|---------------------|-------------| | ||
| | v1.31 | [#6073](https://github.com/dotnet/extensions/pull/6073) | Update to v1.31 conventions | | ||
| | v1.34 | [#6466](https://github.com/dotnet/extensions/pull/6466) | Update to v1.34 conventions | | ||
| | v1.35 | [#6557](https://github.com/dotnet/extensions/pull/6557) | Update to v1.35 conventions | | ||
| | v1.36 | [#6579](https://github.com/dotnet/extensions/pull/6579) | Bump version reference to v1.36 (CCA) | | ||
| | v1.37 | [#6767](https://github.com/dotnet/extensions/pull/6767) | Update to v1.37 conventions | | ||
| | v1.38 | [#6829](https://github.com/dotnet/extensions/pull/6829) | Update to v1.38 conventions | | ||
| | v1.38 update | [#6981](https://github.com/dotnet/extensions/pull/6981) | Additional v1.38 changes | | ||
| | v1.39 | [#7274](https://github.com/dotnet/extensions/pull/7274) | Bump version reference to v1.39 (CCA) | | ||
| | v1.40 | [#7322](https://github.com/dotnet/extensions/pull/7322) | Update to v1.40 conventions (CCA audit) | | ||
|
|
||
| ### Feature-Specific PRs (v1.38–v1.40 era) | ||
|
|
||
| These PRs implemented specific gen-ai convention features rather than being tied to a single version bump: | ||
|
|
||
| | PR | Feature | Convention Source | | ||
| |----|---------|-------------------| | ||
| | [#7240](https://github.com/dotnet/extensions/pull/7240) | Server-side tool call attributes | v1.37+ | | ||
| | [#7241](https://github.com/dotnet/extensions/pull/7241) | Metric computation fix | Bug fix | | ||
| | [#7325](https://github.com/dotnet/extensions/pull/7325) | Streaming metrics (time_to_first_chunk, time_per_output_chunk) | v1.39 | | ||
| | [#7379](https://github.com/dotnet/extensions/pull/7379) | Exception event recording (gen_ai.client.operation.exception) | v1.40 | | ||
| | [#7382](https://github.com/dotnet/extensions/pull/7382) | invoke_workflow operation name | v1.40 | | ||
|
|
||
| ## Authorship Pattern Evolution | ||
|
|
||
| - **v1.29–v1.37**: Human-authored PRs by @stephentoub — terse PR descriptions, often just links | ||
| - **v1.36 onwards**: Copilot Coding Agent (CCA) introduced — detailed prompts in PR body | ||
| - **v1.39–v1.40**: Primarily CCA-authored with structured prompts or audit tables | ||
| - **v1.40 feature PRs**: Gold-standard CCA prompts (#7379, #7382) with Background → Required Changes → Tests structure | ||
|
|
||
| ## Typical Change Patterns by Release | ||
|
|
||
| ### Version-only releases (v1.36, v1.39) | ||
| - Only doc comment version bump needed | ||
| - Minimal code changes | ||
| - Quick turnaround | ||
|
|
||
| ### Attribute addition releases (v1.31, v1.34, v1.35, v1.37, v1.38) | ||
| - New constants in `OpenTelemetryConsts.cs` | ||
| - New attribute emission in one or more OpenTelemetry* clients | ||
| - Test updates | ||
| - Version bump | ||
|
|
||
| ### Behavioral change releases (v1.40 features) | ||
| - New code patterns (exception recording, streaming metrics) | ||
| - May require shared infrastructure (`Common/` classes) | ||
| - More extensive test changes | ||
| - Often split into multiple PRs per feature | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.