Repository navigation
fix(sse): send the sanitized error body on non-streaming provider failures #19097
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| - **fix(sse):** non-streaming failures no longer relay the raw upstream error response to the client; the sanitized error body is sent and upstream headers like `Set-Cookie` are dropped while the `Retry-After` hint is kept. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -474,12 +474,15 @@ export async function runNonStreamingProviderLeg( | |
| outcome.result.status, | ||
| formatted, | ||
| undefined, | ||
| null, | ||
| // Keep the upstream retry hint on the sanitized response. | ||
| outcome.result.response ? parseRetryAfterMs(outcome.result.response) : null, | ||
| outcome.result.errorCode, | ||
| outcome.result.errorType, | ||
| { passthrough: input.sourceFormat === "claude" } | ||
| ), | ||
| response: outcome.result.response, | ||
| // Never relay the raw upstream Response (body, Set-Cookie, other headers) to the | ||
| // client: legError() above built the sanitized one. Upstream headers stay on | ||
| // `upstreamHeaders` for server-side classification only. | ||
|
Comment on lines
480
to
+485
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| rawMessage: outcome.result.rawMessage || outcome.result.error, | ||
| upstreamErrorBody: outcome.result.upstreamErrorBody, | ||
| upstreamHeaders: outcome.result.upstreamHeaders ?? outcome.result.response?.headers, | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| // A non-streaming provider failure must reach the client as the sanitized error body, | ||
| // never as the raw upstream Response (body fields, internal paths, Set-Cookie, other headers). | ||
| import assert from "node:assert/strict"; | ||
| import test from "node:test"; | ||
| import { createChatPipelineHarness } from "../integration/_chatPipelineHarness.ts"; | ||
|
|
||
| const h = await createChatPipelineHarness("nonstream-provider-error-sanitized"); | ||
| const originalFetch = globalThis.fetch; | ||
|
|
||
| const LEAKY_BODY = { | ||
| error: { | ||
| message: "upstream failed at /srv/app/node_modules/foo/index.js:10", | ||
| type: "invalid_request_error", | ||
| internal_trace: "TRACE-LEAK-MARKER", | ||
| }, | ||
| }; | ||
|
|
||
| test.afterEach(() => { | ||
| globalThis.fetch = originalFetch; | ||
| }); | ||
| test.after(() => h.cleanup()); | ||
|
|
||
| for (const stream of [true, false]) { | ||
| test(`stream=${stream}: upstream 400 body and headers do not reach the client`, async () => { | ||
| await h.resetStorage(); | ||
| globalThis.fetch = (async () => | ||
| new Response(JSON.stringify(LEAKY_BODY), { | ||
| status: 400, | ||
| headers: { | ||
| "content-type": "application/json", | ||
| "x-upstream-internal": "HDR-LEAK-MARKER", | ||
| "set-cookie": "sid=UPSTREAM-COOKIE; Path=/", | ||
| }, | ||
| })) as typeof fetch; | ||
| await h.seedConnection("openai", { name: "sanitize-400" }); | ||
|
|
||
| const res = await h.handleChat( | ||
| h.buildRequest({ | ||
| body: { model: "openai/gpt-4o-mini", stream, messages: [{ role: "user", content: "hi" }] }, | ||
| }) | ||
| ); | ||
| const text = await res.text(); | ||
|
|
||
| assert.equal(res.status, 400); | ||
| assert.ok( | ||
| !text.includes("TRACE-LEAK-MARKER"), | ||
| `raw upstream body field reached the client: ${text}` | ||
| ); | ||
| assert.ok(!text.includes("node_modules"), `internal path reached the client: ${text}`); | ||
| assert.equal(res.headers.get("set-cookie"), null, "upstream Set-Cookie reached the client"); | ||
| assert.equal( | ||
| res.headers.get("x-upstream-internal"), | ||
| null, | ||
| "upstream header reached the client" | ||
| ); | ||
| assert.equal(JSON.parse(text).error?.type, "invalid_request_error"); | ||
| }); | ||
| } | ||
|
|
||
| test("stream=false: upstream Retry-After still reaches the client on a 429", async () => { | ||
| await h.resetStorage(); | ||
| globalThis.fetch = (async () => | ||
| new Response(JSON.stringify({ error: { message: "slow down", type: "rate_limit_error" } }), { | ||
| status: 429, | ||
| headers: { "content-type": "application/json", "retry-after": "7" }, | ||
| })) as typeof fetch; | ||
| await h.seedConnection("openai", { name: "sanitize-429" }); | ||
|
|
||
| const res = await h.handleChat( | ||
| h.buildRequest({ | ||
| body: { | ||
| model: "openai/gpt-4o-mini", | ||
| stream: false, | ||
| messages: [{ role: "user", content: "hi" }], | ||
| }, | ||
| }) | ||
| ); | ||
| await res.text(); | ||
|
|
||
| assert.equal(res.status, 429); | ||
| assert.ok(res.headers.get("retry-after"), "Retry-After hint was dropped"); | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Preserve the retry delay from status restatement.
When agentrouter returns a quota error as HTTP 403 without
Retry-After,applyStatusRestatementchanges it to 429 and supplies a 60-second retry delay. The pipeline does not carry that delay inoutcome.result, soparseRetryAfterMs(outcome.result.response)returnsnull. The rebuilt 429 response then omits the retry hint used by clients and cooldown classification. Carry the restatedretryAfterMsthroughProviderExecutionOutcome, and use it before parsing the upstream header. (raw.githubusercontent.com)