fix(openai): account cached tokens for zhipu_v4 usage

[RedwindA]

PR 类型

• Bug 修复
• 新功能
• 文档更新
• 其他

PR 是否包含破坏性更新？

• 是
• 否

PR 描述

为 Zhipu V4 渠道在通用 OpenAI适配器中新增缓存 token 处理逻辑，确保从usage 结构中正确提取 cached_tokens 并回写到
prompt_tokens_details。兼容流式与非流式响应，并在渠道元信息中启用 StreamOptions支持，使缓存命中能正确计费。

Summary by CodeRabbit

• New Features

  • Enabled streaming for the Zhipu v4 channel.

• Bug Fixes

  • Improved accuracy of token usage reporting across providers (OpenAI, OpenRouter, DeepSeek, Zhipu v4).
  • Normalizes cached token counts after responses, ensuring consistent metrics.
  • Automatically extracts cached token data from provider responses when available.
  • Correctly accounts for prompt cache hits to reflect true usage.

[coderabbitai]

Walkthrough

Adds a usage post-processing hook to normalize cached token counts after initial usage computation, with body-based extraction for specific channels (DeepSeek, Zhipu_v4). Integrates this into OpenAI/OpenRouter flows and updates streaming support to include Zhipu_v4.

Changes

Cohort / File(s)	Summary of Changes
Usage post-processing and cached token extraction relay/channel/openai/relay-openai.go	Introduces applyUsagePostProcessing and extractCachedTokensFromBody; populates PromptTokensDetails.CachedTokens from channel-specific fields or response body; applies post-processing across relevant code paths, including Zhipu_v4 handling.
Streaming capability update relay/common/relay_info.go	Marks ChannelTypeZhipu_v4 as supported for streaming in streamSupportedChannels.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Client
  participant Relay as Relay Handler
  participant Channel as Channel Impl
  participant Upstream as Upstream API

  Client->>Relay: Request (chat/completions)
  Relay->>Upstream: Forward request
  Upstream-->>Relay: Response (body, usage)
  Relay->>Channel: Compute initial usage
  note over Relay,Channel: New step: applyUsagePostProcessing
  Channel->>Channel: If DeepSeek/Zhipu_v4: adjust CachedTokens
  alt CachedTokens missing
    Channel->>Channel: Extract from body (usage.prompt_tokens_details.cached_tokens<br/>or usage.cached_tokens or prompt_cache_hit_tokens)
  end
  Channel-->>Relay: Normalized usage
  Relay-->>Client: Response (usage with CachedTokens)

  %% Styling notes
  rect rgba(230, 245, 255, 0.4)
  note right of Channel: New/changed logic: post-processing & body extraction
  end

Loading

sequenceDiagram
  autonumber
  actor Client
  participant Relay as Relay (Streaming)
  participant Zhipu as Zhipu_v4 API

  Client->>Relay: Streaming request
  Relay->>Zhipu: Initiate stream
  Zhipu-->>Relay: Streamed chunks
  Relay->>Relay: Post-process usage on finalization (Zhipu_v4 supported)
  Relay-->>Client: Stream + final usage (with CachedTokens if extractable)

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• fix: responses cache token 未计费 #1509 — Also modifies cached token handling logic with nil checks and conditional assignment; closely aligned with this PR’s post-processing approach.

Poem

I thump my paws at token streams,
Counting crumbs of cached dreams.
From bodies deep, I glean the clues,
To fill the gaps in usage news.
Zhipu hums, the numbers flow—
A rabbit nods: “Now metrics show.” 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title Check	✅ Passed	The title accurately summarizes the primary change by indicating a fix to account for cached tokens in the Zhipu_v4 channel of the OpenAI adapter, matching the bug fix described in the PR. It follows conventional commit style, is concise and directly reflects the implementation without extraneous detail. It clearly conveys the purpose of the changeset so a teammate viewing the history understands the core update.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (2)

relay/channel/openai/relay-openai.go (2)

637-658: Consider documenting the precedence order for cached token extraction.

The logic correctly handles multiple sources for cached tokens, but the precedence order for Zhipu_v4 (InputTokensDetails → body extraction → PromptCacheHitTokens) isn't immediately obvious.

Add a brief comment above the Zhipu_v4 case:

 	case constant.ChannelTypeZhipu_v4:
+		// Precedence: InputTokensDetails.CachedTokens > body extraction > PromptCacheHitTokens
 		if usage.PromptTokensDetails.CachedTokens == 0 {

---

660-689: Consider adding debug logging for unmarshal failures.

The function silently returns (0, false) when JSON unmarshaling fails. While this is acceptable for best-effort extraction, adding debug logging could help diagnose issues when cached tokens aren't extracted as expected.

Add debug logging for unmarshal errors:

 	if err := json.Unmarshal(body, &payload); err != nil {
+		if common.DebugEnabled {
+			common.LogDebug(fmt.Sprintf("Failed to extract cached tokens from response body: %v", err))
+		}
 		return 0, false
 	}

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a610ef4 and f930cdb.

📒 Files selected for processing (2)

• relay/channel/openai/relay-openai.go (3 hunks)
• relay/common/relay_info.go (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

relay/channel/openai/relay-openai.go (3)

dto/openai_response.go (1)

• Usage (221-234)

relay/common/relay_info.go (1)

• RelayInfo (74-121)

constant/channel.go (2)

• ChannelTypeDeepSeek (43-43)
• ChannelTypeZhipu_v4 (30-30)

relay/common/relay_info.go (1)

constant/channel.go (1)

• ChannelTypeZhipu_v4 (30-30)

🔇 Additional comments (4)

relay/common/relay_info.go (1)

264-264: LGTM! Correctly enables stream options support for Zhipu_v4.

Adding ChannelTypeZhipu_v4 to the streamSupportedChannels map enables the SupportStreamOptions flag for this channel, which is essential for proper cached token billing in streaming responses.

relay/channel/openai/relay-openai.go (3)

233-233: LGTM! Correctly applies post-processing with response body.

The non-streaming handler correctly passes the responseBody to enable cached token extraction from the JSON payload when other fields are unavailable.

---

633-633: LGTM! Consistent post-processing in usage handler.

Correctly applies the same post-processing pattern as the main handler.

---

168-169: Cached token extraction for Zhipu_v4 streaming is already handled. The streaming handler in relay_responses.go populates usage.PromptTokensDetails.CachedTokens from InputTokensDetails, so calling applyUsagePostProcessing(nil) safely captures cached tokens.