Skip to content

fix(agent-core): only send prompt_cache_key to official OpenAI endpoints - #2240

Open
creatiVision wants to merge 1 commit into
MoonshotAI:mainfrom
creatiVision:fix/latest-with-prompt-cache
Open

fix(agent-core): only send prompt_cache_key to official OpenAI endpoints#2240
creatiVision wants to merge 1 commit into
MoonshotAI:mainfrom
creatiVision:fix/latest-with-prompt-cache

Conversation

@creatiVision

@creatiVision creatiVision commented Jul 27, 2026

Copy link
Copy Markdown

Related Issue

Tracking / backup of #2203 (original by @B143KC47).
Resolves #2166.
Related: #2611 (Azure Foundry 400 rejection), #2761.

If #2203 is merged first, this PR can be closed with no further action.

Problem

Since v0.29.0 (#1970), every OpenAI-compatible provider receives the session prompt_cache_key in the request body. Strictly-validating endpoints (e.g. NVIDIA NIM at integrate.api.nvidia.com (#2166) or Azure Foundry (#2611)) reject the unknown parameter with:

400 Validation: Unsupported parameter(s): `prompt_cache_key`

That makes custom openai / openai_responses providers unusable for agent sessions.

What changed

Same fix as #2203, rebased onto current main:

  • Only send prompt_cache_key when the effective base URL targets the official OpenAI API (api.openai.com or *.api.openai.com, or unset → client default).
  • v1: gate in toKosongProviderConfig for openai and openai_responses (isOfficialOpenAIBaseUrl).
  • v2: same gate on the bare cacheKeyprompt_cache_key fallback in OpenAI chat-completions and Responses bases.
  • Kimi protocol branch untouched (Kimi endpoints support the field).
  • Tests: custom endpoint omits the field; official endpoint keeps it.

Checklist

@changeset-bot

changeset-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 30e00d4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@moonshot-ai/agent-core-v2 Patch
@moonshot-ai/kimi-code Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@creatiVision creatiVision mentioned this pull request Jul 27, 2026
5 tasks
@creatiVision
creatiVision force-pushed the fix/latest-with-prompt-cache branch 2 times, most recently from c313025 to 6c9fbe6 Compare August 9, 2026 10:44
@creatiVision
creatiVision force-pushed the fix/latest-with-prompt-cache branch 5 times, most recently from 9cba0aa to d18b4de Compare August 20, 2026 20:11
@creatiVision
creatiVision force-pushed the fix/latest-with-prompt-cache branch 2 times, most recently from 8f1a867 to 2024ea0 Compare August 28, 2026 07:25
@TaylorTWBrown

Copy link
Copy Markdown

I'd love to see this merged soon. I can't use Kimi + Kimi Code on Azure as it is.

@creatiVision

Copy link
Copy Markdown
Author

Rebased and synced with latest main. The merge conflicts resulting from the recent kosong package extraction under packages/agent-core-v2 have been resolved by dropping the obsolete packages/agent-core-v2/src/kosong files.

The PR diff is now clean and focused strictly on gating prompt_cache_key behind isOfficialOpenAIBaseUrl(baseUrl) in packages/agent-core/src/session/provider-manager.ts for both openai and openai_responses providers, along with comprehensive unit tests in runtime-provider.test.ts. All test suites pass.

@creatiVision
creatiVision force-pushed the fix/latest-with-prompt-cache branch from 475f680 to 30e00d4 Compare September 7, 2026 07:15
@creatiVision

Copy link
Copy Markdown
Author

Rebased and synced with latest main. With upstream's removal of the legacy packages/agent-core engine (#3542), the gating logic has been cleanly applied to the active packages/agent-core-v2 engine:

  1. packages/agent-core-v2/src/human/llm/requester/bases/openai-base-url.ts: Implements isOfficialOpenAIBaseUrl() helper targeting api.openai.com / *.api.openai.com (or default unset).
  2. packages/agent-core-v2/src/human/llm/requester/bases/openai/format.ts & openai-responses/format.ts: Gates prompt_cache_key behind isOfficialOpenAIBaseUrl(ctx.model.baseUrl) on fallback when no trait hook overrides it, resolving HTTP 400 rejections on strict third-party endpoints like NVIDIA NIM (Error with models not supporting prompt cache #2166) and Azure Foundry (MS Foundry + Kimi Code 2.7 - 400 Unrecognized request argument supplied: prompt_cache_key #2611).
  3. packages/agent-core-v2/src/human/test/llm/cache-key.test.ts: Verified with unit tests that prompt_cache_key is sent to official endpoints and omitted for third-party endpoints (all tests green, no comments in agent-core-v2).
  4. Changeset included.

All merge conflicts are resolved and the PR is clean, mergeable, and ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Error with models not supporting prompt cache

2 participants