Skip to content

fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field - #65678

Merged
teknium1 merged 1 commit into
mainfrom
fix/deepseek-cache-hit-tokens
Jul 16, 2026
Merged

fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field#65678
teknium1 merged 1 commit into
mainfrom
fix/deepseek-cache-hit-tokens

Conversation

@teknium1

@teknium1 teknium1 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Direct DeepSeek API sessions now recognize cache hits: normalize_usage() and the chat-completions transport both read DeepSeek's native top-level usage.prompt_cache_hit_tokens field. Fixes #61871.

Root cause: DeepSeek's own API (api.deepseek.com) reports context-cache hits as top-level prompt_cache_hit_tokens / prompt_cache_miss_tokens (prompt_tokens = hit + miss), not OpenAI's nested prompt_tokens_details.cached_tokens. No code path read those fields (repo-wide grep: zero matches), so direct DeepSeek sessions always showed 0 cache-hit tokens — invisible in accounting, 0% in the cache display, and billed at the full input rate instead of the 50x-cheaper cache-read rate.

Changes

  • agent/usage_pricing.py normalize_usage(): chat-completions branch falls back to prompt_cache_hit_tokens when the nested shape is absent; input tokens correctly derive as the cache-miss bucket.
  • agent/transports/chat_completions.py extract_cache_stats(): same fallback for the live 💾 cache display.
  • Precedence: the OpenAI nested value wins when both shapes are present (proxies), so nothing double-reads.
  • Tests: both layers — native shape recognized, nested-wins precedence, existing OpenAI/Anthropic-proxy shapes untouched.

Validation

Result
Offline E2E (real DeepSeek payload shape) 2000 prompt tokens → 500 input / 1500 cache-read / 400 out; estimated at 2026-07 rates
tests/agent/test_usage_pricing.py + transports/test_chat_completions.py 117/117 passed

Infographic

deepseek-cache-fields

@teknium1
fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field
6606b8a 
DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.

Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).

Fixes #61871.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/deepseek DeepSeek API P3 Low — cosmetic, nice to have labels Jul 16, 2026
tonydwb
tonydwb reviewed Jul 16, 2026
View reviewed changes

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Comment

Overview

  • Usage fix: read DeepSeek's native prompt_cache_hit_tokens cache field
  • 82 additions, 5 deletions

Assessment

  • Targeted fix to read a specific DeepSeek cache field
  • No obvious security concerns
  • Clean diff

Reviewed by Hermes Agent

@teknium1
teknium1 merged commit 03c0b00 into main Jul 16, 2026
33 checks passed
@teknium1
teknium1 deleted the fix/deepseek-cache-hit-tokens branch July 16, 2026 14:29
@teknium1 teknium1 mentioned this pull request Jul 16, 2026
Closed
23 tasks
@DavidMetcalfe DavidMetcalfe mentioned this pull request Jul 16, 2026
Open
@teknium1 teknium1 mentioned this pull request Jul 18, 2026
Closed
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@teknium1
fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field (
3846185 
NousResearch#65678)

DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.

Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).

Fixes NousResearch#61871.
mehmetkr-31 added a commit to mehmetkr-31/hermes-agent that referenced this pull request Jul 31, 2026
@mehmetkr-31 @claude
fix(agent): read Kimi/Moonshot top-level cached_tokens in normalize_u…
8b62fef 
…sage (NousResearch#65722)

Kimi/Moonshot's native API (api.moonshot.cn / .ai) reports context-cache hits
as a top-level ``usage.cached_tokens``. The chat-completions branch of
normalize_usage() walks a fallback chain of
prompt_tokens_details.cached_tokens -> cache_read_input_tokens ->
prompt_cache_hit_tokens; none of those names match, so direct Kimi sessions
normalized to cache_read_tokens=0. The hits were invisible in accounting and
the cached prefix was billed at the full input rate.

Appended as the last link in that chain, so it only fills a genuine zero and
cannot override a provider that reports the nested OpenAI shape or DeepSeek's
prompt_cache_hit_tokens.

Rebuilt on current main rather than rebased — the branch was ~3400 commits
behind. The DeepSeek half of the original branch is dropped: 03c0b00
(NousResearch#65678) landed prompt_cache_hit_tokens on main, so this is Kimi-only as the
review asked. The scripts/release.py addition to the frozen LEGACY_AUTHOR_MAP
is dropped too; contributors/emails/mehmet.kar@std.yildiz.edu.tr already
exists on main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@teknium1
fix(usage): read DeepSeek's native prompt_cache_hit_tokens cache field (
9978825 
NousResearch#65678)

DeepSeek's own API (api.deepseek.com) reports context-cache hits as
top-level usage.prompt_cache_hit_tokens / prompt_cache_miss_tokens
(prompt_tokens = hit + miss), not the OpenAI nested
prompt_tokens_details.cached_tokens shape. Neither normalize_usage()
nor the chat_completions transport's extract_cache_stats() read those
fields, so direct DeepSeek sessions always showed 0 cache-hit tokens:
invisible in accounting, mis-billed at the full input rate, and 0%
cache display.

Both layers now fall back to prompt_cache_hit_tokens when the nested
shape is absent; the nested value wins when both are present (proxies).

Fixes NousResearch#61871.
teknium1 pushed a commit that referenced this pull request Aug 14, 2026
@mehmetkr-31 @claude @teknium1
fix(agent): read Kimi/Moonshot top-level cached_tokens in normalize_u…
5b82e69 
…sage (#65722)

Kimi/Moonshot's native API (api.moonshot.cn / .ai) reports context-cache hits
as a top-level ``usage.cached_tokens``. The chat-completions branch of
normalize_usage() walks a fallback chain of
prompt_tokens_details.cached_tokens -> cache_read_input_tokens ->
prompt_cache_hit_tokens; none of those names match, so direct Kimi sessions
normalized to cache_read_tokens=0. The hits were invisible in accounting and
the cached prefix was billed at the full input rate.

Appended as the last link in that chain, so it only fills a genuine zero and
cannot override a provider that reports the nested OpenAI shape or DeepSeek's
prompt_cache_hit_tokens.

Rebuilt on current main rather than rebased — the branch was ~3400 commits
behind. The DeepSeek half of the original branch is dropped: 03c0b00
(#65678) landed prompt_cache_hit_tokens on main, so this is Kimi-only as the
review asked. The scripts/release.py addition to the frozen LEGACY_AUTHOR_MAP
is dropped too; contributors/emails/mehmet.kar@std.yildiz.edu.tr already
exists on main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@tonydwb tonydwb tonydwb left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/deepseek DeepSeek API type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: direct DeepSeek API cache-hit tokens never extracted — only OpenAI/OpenRouter prompt_tokens_details.cached_tokens is read

3 participants

@teknium1 @tonydwb @alt-glitch