Skip to content

fix: preserve prompt_cache_key in Responses API, escape \n in tagContent (#517, #515) #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
diegosouzapw merged 1 commit into from
Mar 21, 2026
Merged

fix: preserve prompt_cache_key in Responses API, escape \n in tagContent (#517, #515) #518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion open-sse/services/combo.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,7 @@ export async function handleComboChat({
// SDKs close the connection on finish_reason, so anything sent after
// that marker is silently dropped.
if (!res.body) return res;
const tagContent = `\n<omniModel>${modelStr}</omniModel>\n`;
const tagContent = `\\n<omniModel>${modelStr}</omniModel>\\n`;

@gemini-code-assist gemini-code-assist Bot Mar 21, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This change fixes the JSON injection for streaming responses by escaping the newline characters. However, this breaks another usage of tagContent in the flush part of the TransformStream (line 514), where it's used in a JSON.stringify call.

With this change, tagContent is a string containing a literal backslash and 'n' (e.g., "\\n..."). When JSON.stringify({ content: tagContent }) is called, the backslash is also escaped, resulting in "\\\\n..." in the JSON output. The client will then parse this as a literal \n string, not a newline character.

To address this, tagContent should contain raw newlines when passed to JSON.stringify. A comprehensive fix would involve defining the raw content and escaping it only for the regex injection. Since that requires changing code outside this diff, a more localized fix would be to un-escape tagContent at line 514 before it's stringified. For example: delta: { content: tagContent.replace(/\\n/g, '\n') }.

Please adjust the implementation to ensure both use cases are handled correctly.

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tagContent is now pre-escaped with \\n, but later you embed it into JSON in two different ways: (1) via raw string replacement into an existing JSON line, and (2) via JSON.stringify(...) in flush(). With the new value, path (2) will double-escape backslashes so clients receive a literal "\n" sequence, while path (1) will be parsed as an actual newline escape, making behavior inconsistent. Consider keeping tagContent as real newlines and using proper JSON escaping for the replacement (e.g., derive an escaped form from JSON.stringify(tagContent)), or otherwise ensure both paths emit the same decoded content.

Copilot uses AI. Check for mistakes.

@chatgpt-codex-connector chatgpt-codex-connector Bot Mar 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid double-escaping the fallback omniModel chunk

When context_cache_protection is enabled and a streamed response never emits a delta.content chunk before flush runs (tool-call-only turns are a common case), this escaped tagContent is fed through JSON.stringify in the fallback chunk builder, so clients receive literal \\n text instead of real newlines. On the next request extractPinnedModel() still finds the tag, but stripModelTags() in open-sse/services/comboAgentMiddleware.ts removes only the tag and leaves "\\n\\n" behind, which then gets forwarded upstream as prompt text. The old raw-newline form trimmed away cleanly, so this change regresses context-cache protection for no-text/tool-only streams.

Useful? React with 👍 / 👎.

const encoder = new TextEncoder();
const decoder = new TextDecoder();
let tagInjected = false;
Expand Down
3 changes: 2 additions & 1 deletion open-sse/translator/helpers/responsesApiHelper.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,11 @@ export function convertResponsesApiFormat(body) {
}

// Cleanup Responses API specific fields
// Note: prompt_cache_key is intentionally preserved — it is used by Codex and other
// providers as a cache-affinity signal. Stripping it breaks prompt caching (#517).
delete result.input;
delete result.instructions;
delete result.include;
delete result.prompt_cache_key;
delete result.store;
delete result.reasoning;

Expand Down
3 changes: 2 additions & 1 deletion open-sse/translator/request/openai-responses.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,10 +227,11 @@ export function openaiResponsesToOpenAIRequest(
});

// Cleanup Responses API specific fields
// Note: prompt_cache_key is intentionally preserved — it is used by Codex and other
// providers as a cache-affinity signal. Stripping it breaks prompt caching (#517).
delete result.input;
delete result.instructions;
delete result.include;
delete result.prompt_cache_key;
delete result.store;
delete result.reasoning;

Expand Down
Loading