fix: include cached tokens in input_tokens - #4080
opieter-aws merged 2 commits into
Conversation
|
Issue: The description lists Anthropic-direct among the disjoint-cache providers this fix corrects, but the shared So in practice only Bedrock Converse (whose Suggestion: Reword the description so it doesn't claim Anthropic-direct is fixed here — either scope it to Bedrock Converse, or explicitly call out Anthropic-direct as a still-open follow-up ( |
|
Assessment: Approve (with one minor description clarification) Clean, well-scoped fix that centralizes the correction in Review Categories
Nice reuse of the existing helper to keep the tracer, meter, and agent token math consistent. |
|
Re-reviewed after the update — code is unchanged; the deltas are the title reclassification and a reworded description. Title One residual nit on the reworded description: it still frames "Bedrock Converse and Anthropic-direct excludes cached tokens" as the problem being corrected, but per the #3546 known limitation this change only actually folds cache into Assessment stands: Approve. |
Description
The TypeScript tracer emitted the provider’s raw inputTokens as
gen_ai.usage.input_tokens, which for Bedrock Converse and Anthropic-direct excludes cached tokens. This violates the OTEL GenAI semantic conventions, whereinput_tokensmust include cached tokens.For subset-cache providers (OpenAI, Gemini) where inputTokens already includes cache, the function returns the value unchanged, so there is no double counting. This brings TS to parity with the Python fix in #3886.
Related Issues
Follows #3886
Documentation PR
No documentation changes required.
Type of Change
Bug fix
Testing
Ran the TypeScript telemetry unit suite in both Node and browser environments (
npx vitest run src/telemetry/__tests__/tracer.test.node.ts src/telemetry/__tests__/meter.test.ts), plusnpm run type-check,npm run lint, andnpx prettier. Added regression coverage for the disjoint-cache path on bothendModelInvokeSpanandendAgentSpan(cached tokens folded intoinput_tokens) and for the subset-cache path (fold-in is a no-op, no double count).hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.**