fix(openai): unify responses upstream header handling

[zhaowenzi]

Description

Problem

/v1/responses used inconsistent upstream header behavior across code paths.

The non-streaming path applied auth explicitly, while the streaming and MCP-assisted paths forwarded incoming request headers more broadly. That caused behavior differences based on stream: true/false or whether MCP tool looping was triggered. In particular, headers like OpenAI-Project could reach the OpenAI upstream on only some /responses paths, and the streaming / MCP paths also missed worker API key fallback when the caller did not provide Authorization.

Solution

Align the streaming and MCP-assisted /responses paths with the existing non-streaming behavior by applying provider-aware auth handling directly at each upstream call site.

The updated behavior is:

• prefer caller auth when present
• fall back to the worker API key when caller auth is absent
• do not forward arbitrary caller headers upstream

This is now applied consistently in:

• non-streaming /responses
• streaming passthrough /responses
• streaming MCP-intercepted /responses
• non-streaming MCP tool loop upstream requests

Provider-aware extraction is preserved, so if these paths hit a non-OpenAI upstream in a single-provider setup, headers like x-api-key / x-goog-api-key still work correctly.

Changes

• updated non-streaming /responses to use provider-aware auth extraction inline
• updated streaming /responses passthrough to use the same auth behavior inline
• updated streaming MCP interception requests back to the upstream model to use the same auth behavior inline
• updated MCP tool loop upstream requests back to the model to use the same auth behavior inline
• added worker API key fallback to the streaming and MCP-assisted /responses paths
• removed the now-unused apply_request_headers helper
• added regression tests covering:
  • worker API key fallback when non-auth headers are present
  • provider-aware extraction for Anthropic-style x-api-key

Test Plan

Before:

curl -i http://localhost:9999/v1/responses \
  -H "Content-Type: application/json" \
  -H "OpenAI-Project: 04022026_02" \
  -H "Authorization: Bearer sk-svcacct-..." \
  -d '{
    "model": "gpt-5.4",
    "input": "hello",
    "stream": true
  }'

Expected before this change:

• streaming /responses could forward OpenAI-Project upstream
• OpenAI could reject the request with:

{
  "error": {
    "message": "OpenAI-Project header should match project for API key",
    "type": "invalid_request_error",
    "code": "mismatched_project",
    "param": null
  },
  "status": 401
}

After:

curl -i http://localhost:9999/v1/responses \
  -H "Content-Type: application/json" \
  -H "OpenAI-Project: 04022026_02" \
  -H "Authorization: Bearer sk-svcacct-..." \
  -d '{
    "model": "gpt-5.4",
    "input": "hello",
    "stream": true
  }'

Expected after this change:

• streaming /responses matches the non-streaming path
• OpenAI-Project is not forwarded upstream on this path
• the request succeeds instead of failing with mismatched_project

Additional verification:

cargo check -p smg
cargo test -p smg header_utils

Checklist

• cargo +nightly fmt passes
• cargo clippy --all-targets --all-features -- -D warnings passes
• (Optional) Documentation updated
• (Optional) Please join us on Slack #sig-smg to discuss, review, and merge PRs

Summary by CodeRabbit

• Refactor
  • Consolidated authentication header extraction and application logic across API provider integrations to ensure consistent credential handling.
  • Enhanced authentication mechanisms for streaming requests and tool-based execution workflows with improved credential propagation.
  • Streamlined internal header management to ensure authentication credentials are properly forwarded through upstream API requests and responses.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 378eadaf-54dd-410e-8adb-e45ef71f3d9a

📥 Commits

Reviewing files that changed from the base of the PR and between 228d84a and 02d60c8.

📒 Files selected for processing (4)

• model_gateway/src/routers/header_utils.rs
• model_gateway/src/routers/openai/mcp/tool_loop.rs
• model_gateway/src/routers/openai/responses/non_streaming.rs
• model_gateway/src/routers/openai/responses/streaming.rs

---

📝 Walkthrough

Walkthrough

This PR refactors header handling by centralizing auth header extraction and request header application through an ApiProvider abstraction. It removes the standalone apply_request_headers() function and threads the worker API key through multiple function signatures to enable consistent provider-scoped header management across tool execution and response handling paths.

Changes

Cohort / File(s)	Summary
Header utilities model_gateway/src/routers/header_utils.rs	Removed public apply_request_headers() function; added two unit tests for extract_auth_header() fallback behavior and Anthropic provider API key preference.
MCP tool execution model_gateway/src/routers/openai/mcp/tool_loop.rs	Updated execute_tool_loop() signature to accept worker_api_key: Option<&String>; switched from conditional apply_request_headers() calls to consistent provider-based header application via ApiProvider.apply_headers().
Response handlers model_gateway/src/routers/openai/responses/non_streaming.rs, model_gateway/src/routers/openai/responses/streaming.rs	Replaced direct apply_request_headers() calls with ApiProvider::from_url() pattern; updated handle_streaming_with_tool_interception() signature to accept worker_api_key: Option<String>; changed header application from conditional to unconditional via provider methods.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• refactor(gateway): extract MCP module from OpenAI responses #730 — Refactored execute_tool_loop location and structure; this PR modifies its signature and internal header handling.
• refactor(mcp): remove requested_servers and introduce ResponsesCallContext #382 — Related refactoring of auth/header propagation and MCP tool-loop signatures with similar worker API key threading patterns.
• feat(gateway): API-key-aware /v1/models with upstream fan-out #698 — Touches upstream header handling and provider-scoped header application logic within the same domain.

Suggested labels

tests, anthropic

Suggested reviewers

• CatherineSue
• key4ng
• slin1237

Poem

🐰 Headers once scattered, now gathered with care,
Through ApiProvider, they glide through the air!
The worker_api_key threads through each call,
Auth flows more cleanly—we've centralized all! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(openai): unify responses upstream header handling' directly corresponds to the main objective of aligning inconsistent header behavior across the /responses code paths (non-streaming, streaming, and MCP-assisted), making upstream header handling consistent via provider-aware auth extraction.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/openai-header

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ce229cfcc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[gemini-code-assist]

Code Review

This pull request refactors header application logic by introducing apply_upstream_auth_headers to replace the more generic apply_request_headers across the OpenAI response and MCP tool loop paths. This change centralizes authentication normalization for external providers. Feedback indicates that the new helper should be updated to support provider-specific authentication headers and to maintain the forwarding of tracing and correlation headers for observability.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@model_gateway/src/routers/header_utils.rs`:
- Around line 350-389: Add a new unit test in header_utils.rs that exercises
apply_upstream_auth_headers with Some(&headers) where headers contains
non-authorization headers but no Authorization header, ensuring the function
does not forward arbitrary headers and instead uses the worker fallback; create
a test named like test_apply_upstream_auth_headers_some_headers_no_caller_auth
(or similar), construct a HeaderMap with e.g. "openai-project" and
"x-custom-header" but no "authorization", call apply_upstream_auth_headers with
that headers Some reference and a worker secret, build the request, and assert
that the resulting request has the worker "authorization" header and does not
contain the other custom headers.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9e9c0e8f-e850-4344-b8d0-bed7ba1f1ee2

📥 Commits

Reviewing files that changed from the base of the PR and between b02ae6b and 6ce229c.

📒 Files selected for processing (4)

• model_gateway/src/routers/header_utils.rs
• model_gateway/src/routers/openai/mcp/tool_loop.rs
• model_gateway/src/routers/openai/responses/non_streaming.rs
• model_gateway/src/routers/openai/responses/streaming.rs

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 228d84a524

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: afdfd0cb2e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 02d60c8b7f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".