Skip to content

fix(gateway): redact stealth provider errors on mid-stream read faults - #3348

Merged
steebchen merged 4 commits into
theopenco:mainfrom
pacocartones:fix/redact-stealth-errors-midstream
Aug 2, 2026
Merged

steebchen merged 4 commits into
theopenco:mainfrom
pacocartones:fix/redact-stealth-errors-midstream

Conversation

@pacocartones

@pacocartones pacocartones commented Jul 31, 2026 •

Copy link
Copy Markdown
Contributor

What & why

The mid-stream read-fault error branch wrote the normalized error to the SSE client verbatim, with no shouldRedactProviderError() check — unlike every sibling branch, which all redact for stealth providers:

  • HTTP-error branch (chat.ts ~8335) → redactedProviderErrorText(...)
  • in-stream OpenAI-compatible error event (chat.ts ~9254)
  • non-streaming path (normalize-client-error.ts)

So a network fault after the stream started (socket reset / partial vendor-branded body) leaked a stealth provider's identity three ways through the client payload built in normalize-streaming-error.ts:

  1. message — "Streaming error: <raw>"
  2. details.cause — extractErrorCause() walks the undici cause chain and embeds the secret host (getaddrinfo ENOTFOUND <host>, connect ECONNREFUSED <host>:443, TLS cert host)
  3. responseText — bufferSnapshot, up to 5000 chars of the raw upstream body (vendor-branded)

This is exactly the class of leak the recent stealth-provider compliance work guards against.

Change

Add an optional redact flag to normalizeStreamingError. chat.ts passes redact: shouldRedactProviderError(usedProvider) on the upstream_read branch. When set, the client payload becomes a status-only body via redactedProviderErrorText(statusCode), dropping message, cause, the buffered body, and the Node-level errorName/errorCode — matching redactErrorDetails on the sibling branches. The internal log payload is left raw (feeds the internal-only log columns, consistent with the other branches and logs.ts).

Non-stealth providers are unaffected (the flag is only set when shouldRedactProviderError is true).

Tests

normalize-streaming-error.spec.ts:

  • redacted variant — asserts the serialized client payload contains none of the vendor identity / secret host / raw message / buffered body, that details.cause/errorName/errorCode are dropped, and that only the gateway-derived status survives; the internal log still carries the raw detail.
  • redact-off path — unchanged.
vitest run apps/gateway/src/chat/tools/normalize-streaming-error.spec.ts → 11/11 pass

Scope note: coverage is at the normalizeStreamingError unit boundary (where the client payload is built and emitted verbatim by the route). An end-to-end harness test (stealth-error-redaction.spec.ts) would need the Postgres/Redis test harness; happy to add it if you'd like.


AI-assisted; reviewed and verified by the author.

Summary by CodeRabbit

  • Bug Fixes

    • Improved error handling for streaming connection failures by hiding provider-identifying details, network diagnostics, and buffered response content from applicable client responses.
    • Preserved gateway status information and full diagnostics for internal troubleshooting.
    • Retained existing error details for scenarios that do not require redaction.
  • Tests

    • Added coverage for redacted and non-redacted streaming errors, including interrupted and malformed responses.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The streaming error normalizer now supports optional redaction. Stealth-provider read errors use this mode. Client SSE responses omit provider details and buffered upstream data while internal diagnostics remain unchanged.

Changes

Streaming error redaction

Layer / File(s) Summary
Redacted streaming error normalization
apps/gateway/src/chat/tools/normalize-streaming-error.ts, apps/gateway/src/chat/tools/normalize-streaming-error.spec.ts
Adds the optional redact option. Redacted payloads expose gateway status details and a sanitized message. Default payloads retain existing error metadata and buffered response text. Tests cover generic failures, terminated streams, default behavior, and internal diagnostics.
Stealth-provider redaction wiring and integration tests
apps/gateway/src/chat/chat.ts, apps/gateway/src/stealth-error-redaction.spec.ts
Enables redaction for stealth-provider streaming read errors. Tests simulate truncated SSE data and verify different client and log output for stealth and non-stealth providers.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Gateway
  participant UpstreamProvider
  participant Client
  Gateway->>UpstreamProvider: request streaming response
  UpstreamProvider-->>Gateway: truncated SSE data and socket read failure
  Gateway->>Gateway: determine provider redaction policy
  Gateway-->>Client: sanitized stealth error or raw non-stealth error
Loading

Possibly related PRs

  • theopenco/llmgateway#3090: The streaming normalizer and call sites extend the related stealth-provider error-redaction work.

Suggested reviewers: steebchen

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: redacting stealth provider errors during mid-stream gateway read faults.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/gateway/src/chat/tools/normalize-streaming-error.ts`:
- Around line 192-209: Update the NormalizedStreamingError client details type
so details.errorName is optional, allowing the redacted branch to include only
statusCode and statusText while preserving the existing undefined runtime value
asserted by the redaction test.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9bad9b54-7fbb-4849-963d-93c313f84016

📥 Commits

Reviewing files that changed from the base of the PR and between 580336c and 4b1c62a.

📒 Files selected for processing (3)
  • apps/gateway/src/chat/chat.ts
  • apps/gateway/src/chat/tools/normalize-streaming-error.spec.ts
  • apps/gateway/src/chat/tools/normalize-streaming-error.ts

Comment thread apps/gateway/src/chat/tools/normalize-streaming-error.ts
The mid-stream read-fault error branch wrote the normalized error to the SSE
client verbatim, with no shouldRedactProviderError() check — unlike every
sibling branch (the HTTP-error branch at chat.ts:8335, the in-stream error
event at chat.ts:9254, and the non-streaming normalize-client-error path),
which all redact for stealth providers.

For a stealth provider that leaked its identity three ways in the client
payload built by normalizeStreamingError: the raw message ("Streaming
error: ..."), details.cause (extractErrorCause walks the undici cause chain and
embeds the secret host via ENOTFOUND/ECONNREFUSED/TLS errors), and responseText
(the buffered upstream body, up to 5000 chars of vendor-branded content).

Add an optional redact flag to normalizeStreamingError; when set (chat.ts passes
shouldRedactProviderError(usedProvider) on the upstream_read branch) the
client payload is reduced to a status-only body via redactedProviderErrorText,
dropping message, cause, buffered body, errorName and errorCode — matching
redactErrorDetails on the sibling branches. The internal log payload is left
raw, consistent with the other branches and logs.ts.

Non-stealth providers are unaffected. Adds unit coverage in
normalize-streaming-error.spec.ts (redacted variant scrubs identity + drops the
Node failure identifiers; redact-off path unchanged; internal log stays raw).
@pacocartones
pacocartones force-pushed the fix/redact-stealth-errors-midstream branch from 4b1c62a to 2aec253 Compare August 2, 2026 10:38
@pacocartones

Copy link
Copy Markdown
Contributor Author

Applied the review nit: NormalizedStreamingError.client.details.errorName is now optional, which matches the redacted branch that only emits statusCode/statusText (this also resolves the TS2322 type error on that branch). The branch has been rebased onto current main; typecheck and the streaming-error spec (11 tests) were re-run and pass.

steebchen and others added 2 commits August 2, 2026 18:08
Adds route-level coverage for the leak this PR fixes: the leaky mock now
kills the socket after a half-written, vendor-branded SSE event, so the
gateway fails inside reader.read() with those raw bytes still in its
buffer. Verified to fail without the redact flag.

Also adds a non-stealth control (same fault, openai) asserting the
buffered body and UND_ERR_SOCKET detail still reach the client, plus
unit coverage for the redacted 502 termination path and an explicit
redact: false case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@steebchen
steebchen enabled auto-merge August 2, 2026 17:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
apps/gateway/src/stealth-error-redaction.spec.ts (1)

306-336: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert internal diagnostic retention in the route test.

This test verifies public-log redaction. It does not verify that the integration path stores TRUNCATED_LEAKY_MARKER and UND_ERR_SOCKET in internalErrorDetails. Reuse expectRedactedLog() and assert both values.

Proposed test update
-		const log = await waitForLogByRequestId(requestId);
-		expect(log.hasError).toBe(true);
-		expectNoLeak(JSON.stringify(log.errorDetails));
+		const log = await expectRedactedLog(requestId);
+		expect(JSON.stringify(log.internalErrorDetails)).toContain(
+			TRUNCATED_LEAKY_MARKER,
+		);
+		expect(JSON.stringify(log.internalErrorDetails)).toContain("UND_ERR_SOCKET");
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/gateway/src/stealth-error-redaction.spec.ts` around lines 306 - 336, The
test around the mid-stream read fault currently checks only public log
redaction; update it to reuse expectRedactedLog() and assert that
log.internalErrorDetails retains both TRUNCATED_LEAKY_MARKER and UND_ERR_SOCKET,
while preserving the existing SSE leak assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@apps/gateway/src/stealth-error-redaction.spec.ts`:
- Around line 306-336: The test around the mid-stream read fault currently
checks only public log redaction; update it to reuse expectRedactedLog() and
assert that log.internalErrorDetails retains both TRUNCATED_LEAKY_MARKER and
UND_ERR_SOCKET, while preserving the existing SSE leak assertions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 31de686d-5711-43a4-bacd-54d9e5fefc12

📥 Commits

Reviewing files that changed from the base of the PR and between 2aec253 and 4e275ca.

📒 Files selected for processing (2)
  • apps/gateway/src/chat/tools/normalize-streaming-error.spec.ts
  • apps/gateway/src/stealth-error-redaction.spec.ts

@steebchen
steebchen added this pull request to the merge queue Aug 2, 2026
Merged via the queue into theopenco:main with commit 797db4c Aug 2, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants