genai: define cached tokens attributes#3163
Merged
joaopgrassi merged 10 commits intoopen-telemetry:mainfrom Jan 27, 2026
Merged
genai: define cached tokens attributes#3163joaopgrassi merged 10 commits intoopen-telemetry:mainfrom
joaopgrassi merged 10 commits intoopen-telemetry:mainfrom
Conversation
alexmojaki
reviewed
Dec 3, 2025
lmolkova
moved this from Untriaged
to Awaiting codeowners approval
in Semantic Conventions Triage
lmolkova
moved this to In Progress
in GenAI Semantic Conventions and Instrumentation libraries
|
@lmolkova are there any concerns here or do you think this can be merged? Would be great to get these added to the spec |
Contributor
|
@eddyerburgh see my review comment, that's a required change |
10 tasks
verdie-g
force-pushed
the
gen-ai-cached-tokens
branch
from
c959bb6 to
c2e4498
Compare
alexmojaki
approved these changes
Dec 19, 2025
|
This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days. |
Contributor
Author
|
@lmolkova could you have a look at this one before it gets automatically closed 🙏 |
|
This PR has been labeled as stale due to lack of activity. It will be automatically closed if there is no further activity over the next 7 days. |
Contributor
|
There's merge conflicts now |
verdie-g
force-pushed
the
gen-ai-cached-tokens
branch
from
c2e4498 to
b86579e
Compare
lmolkova
reviewed
Jan 19, 2026
3 tasks
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
verdie-g
force-pushed
the
gen-ai-cached-tokens
branch
from
fc9db01 to
1c01bce
Compare
lmolkova
approved these changes
Jan 26, 2026
github-project-automation
Bot
moved this from Awaiting codeowners approval
to Needs More Approval
in Semantic Conventions Triage
Member
|
@verdie-g thanks for the contribution, everything looks great. I think you need to regenerate markdown tables and it'd be good to go! |
lmolkova
moved this from Needs More Approval
to Ready to be Merged
in Semantic Conventions Triage
github-project-automation
Bot
moved this from In Progress
to Done
in GenAI Semantic Conventions and Instrumentation libraries
stephentoub
reviewed
Jan 27, 2026
| note: Add cache token attributes and provider-specific normalization guidance for GenAI usage metrics | ||
| issues: [1959] | ||
| subtext: | | ||
| - Add `gen_ai.usage.cache_read.input_tokens` attribute for tokens served from provider cache |
There was a problem hiding this comment.
Should these be recorded in metrics as well? Assuming yes, what should the gen_ai.token.type be?
Member
There was a problem hiding this comment.
@stephentoub not yet, there is no metric definition for it (no type defined). There are probably some design flaws we'll need to fix to make it happen. Created https://github.com/open-telemetry/semantic-conventions/issues/3341 so we don't forget
4 tasks
rogeriochaves
added a commit
to langwatch/langwatch
that referenced
this pull request
Feb 8, 2026
…ventions Add support for extracting cache_read_input_tokens, cache_creation_input_tokens, and reasoning_tokens from OTEL span attributes in both the Elasticsearch ingestion path (otel.traces.ts) and the ClickHouse-to-ES mapper (span.mapper.ts). - Extract `gen_ai.usage.cache_read.input_tokens` and `gen_ai.usage.cache_creation.input_tokens` per official OTEL GenAI semconv (open-telemetry/semantic-conventions#3163) - Extract `gen_ai.usage.reasoning_tokens` (Traceloop/OpenLLMetry convention) - Aggregate cache and reasoning tokens at trace level in metrics computation - Add Elasticsearch schema fields and migration for the new token metrics - Clean up redundant `gen_ai.usage.total_tokens` from params - Remove leftover debug console.logs from otel.metrics.ts
rogeriochaves
added a commit
to langwatch/langwatch
that referenced
this pull request
Feb 9, 2026
* feat: support gen_ai.input/output.messages and parts pattern in OTEL traces
Map gen_ai.input.messages and gen_ai.output.messages (OTEL GenAI semantic
conventions) to LangWatch input/output fields. Previously these ended up
in params with input: null and output: null.
- Extract gen_ai.input/output.messages directly as chat_messages without
Zod validation to avoid stripping provider-specific content blocks
(Anthropic tool_use/tool_result, pi-ai parts, etc.)
- Support the "parts" pattern (Vercel AI SDK / pi-ai) natively in
ChatMessage type alongside "content"
- Handle content blocks with {type:"text", content:"..."} in addition
to {type:"text", text:"..."} for text extraction
- Widen LLM type detection to include CLIENT span kind (OTEL GenAI spec)
- Increase params truncation from 32KB to 128KB
- Extract system instructions from pre-existing gen_ai.input.messages
in ClickHouse pipeline
* feat: extract cache and reasoning tokens from OTEL GenAI semantic conventions
Add support for extracting cache_read_input_tokens, cache_creation_input_tokens,
and reasoning_tokens from OTEL span attributes in both the Elasticsearch
ingestion path (otel.traces.ts) and the ClickHouse-to-ES mapper (span.mapper.ts).
- Extract `gen_ai.usage.cache_read.input_tokens` and
`gen_ai.usage.cache_creation.input_tokens` per official OTEL GenAI semconv
(open-telemetry/semantic-conventions#3163)
- Extract `gen_ai.usage.reasoning_tokens` (Traceloop/OpenLLMetry convention)
- Aggregate cache and reasoning tokens at trace level in metrics computation
- Add Elasticsearch schema fields and migration for the new token metrics
- Clean up redundant `gen_ai.usage.total_tokens` from params
- Remove leftover debug console.logs from otel.metrics.ts
* fix: add missing monitors.getById mock in OnlineEvaluationDrawer test helpers
The useContext mock was missing monitors.getById.invalidate, causing
3 unhandled TypeError exceptions during update mutation tests.
* fix: use long type for cache token fields in ES schema and migration
The field was already mapped as long in ES (from OTEL intValue ingestion),
so the migration must use long to avoid illegal_argument_exception.
* feat: stop deleting token attrs from params and display cache tokens in drawer
- Stop deleting gen_ai.usage.* token fields from span params so they
remain visible in the trace details
- Display cache_read and cache_creation tokens in the SpanDetails drawer
alongside prompt/completion/reasoning tokens
sergioestebance
pushed a commit
to langwatch/langwatch
that referenced
this pull request
Feb 9, 2026
* feat: support gen_ai.input/output.messages and parts pattern in OTEL traces
Map gen_ai.input.messages and gen_ai.output.messages (OTEL GenAI semantic
conventions) to LangWatch input/output fields. Previously these ended up
in params with input: null and output: null.
- Extract gen_ai.input/output.messages directly as chat_messages without
Zod validation to avoid stripping provider-specific content blocks
(Anthropic tool_use/tool_result, pi-ai parts, etc.)
- Support the "parts" pattern (Vercel AI SDK / pi-ai) natively in
ChatMessage type alongside "content"
- Handle content blocks with {type:"text", content:"..."} in addition
to {type:"text", text:"..."} for text extraction
- Widen LLM type detection to include CLIENT span kind (OTEL GenAI spec)
- Increase params truncation from 32KB to 128KB
- Extract system instructions from pre-existing gen_ai.input.messages
in ClickHouse pipeline
* feat: extract cache and reasoning tokens from OTEL GenAI semantic conventions
Add support for extracting cache_read_input_tokens, cache_creation_input_tokens,
and reasoning_tokens from OTEL span attributes in both the Elasticsearch
ingestion path (otel.traces.ts) and the ClickHouse-to-ES mapper (span.mapper.ts).
- Extract `gen_ai.usage.cache_read.input_tokens` and
`gen_ai.usage.cache_creation.input_tokens` per official OTEL GenAI semconv
(open-telemetry/semantic-conventions#3163)
- Extract `gen_ai.usage.reasoning_tokens` (Traceloop/OpenLLMetry convention)
- Aggregate cache and reasoning tokens at trace level in metrics computation
- Add Elasticsearch schema fields and migration for the new token metrics
- Clean up redundant `gen_ai.usage.total_tokens` from params
- Remove leftover debug console.logs from otel.metrics.ts
* fix: add missing monitors.getById mock in OnlineEvaluationDrawer test helpers
The useContext mock was missing monitors.getById.invalidate, causing
3 unhandled TypeError exceptions during update mutation tests.
* fix: use long type for cache token fields in ES schema and migration
The field was already mapped as long in ES (from OTEL intValue ingestion),
so the migration must use long to avoid illegal_argument_exception.
* feat: stop deleting token attrs from params and display cache tokens in drawer
- Stop deleting gen_ai.usage.* token fields from span params so they
remain visible in the trace details
- Display cache_read and cache_creation tokens in the SpanDetails drawer
alongside prompt/completion/reasoning tokens
|
This PR contains changes to area(s) that do not have an active SIG/project and will be auto-closed:
Such changes may be rejected or put on hold until a new SIG/project is established. Please refer to the Semantic Convention Areas |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes open-telemetry/semantic-conventions-genai#23
Changes
This PR adds two new GenAI attributes to represent provider-level prompt caching:
gen_ai.usage.cache_read_input_tokensgen_ai.usage.cache_creation_input_tokensIt also updates the description of gen_ai.usage.input_tokens to state that it must include cached tokens. OpenAI and Vertex AI already count cached tokens in input_tokens, while Anthropic excludes them.
Merge requirement checklist
[chore]