Skip to content

fix(responses): forward client headers on response management routes - #35735

Open
devin-ai-integration[bot] wants to merge 4 commits into
litellm_internal_stagingfrom
litellm_forward_client_headers_responses_management_routes
Open

fix(responses): forward client headers on response management routes#35735
devin-ai-integration[bot] wants to merge 4 commits into
litellm_internal_stagingfrom
litellm_forward_client_headers_responses_management_routes

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • get, delete, cancel, list-input-items, compact merge the headers kwarg
  • explicit extra_headers still wins, case-insensitively

Relevant issues

Follow-up to #34531, requested in review

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy with general_settings.forward_client_headers_to_llm_api: true and gpt-5.6-sol (real OpenAI calls); api_base points at a tiny transparent relay in front of https://api.openai.com that logs the x-my-* headers the provider actually receives, since the header is invisible in the response body

RESP=$(curl -s localhost:4000/v1/responses \
  -H 'content-type: application/json' \
  -H 'x-my-new-header: hello-from-client' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{"model":"gpt-5.6-sol","input":"say hi","store":true}')
ID=$(echo "$RESP" | python -c 'import sys,json;print(json.load(sys.stdin)["id"])')

curl -s -o /dev/null "localhost:4000/v1/responses/$ID"             -H 'x-my-new-header: hello-from-client' -H 'Authorization: Bearer sk-1234'
curl -s -o /dev/null "localhost:4000/v1/responses/$ID/input_items" -H 'x-my-new-header: hello-from-client' -H 'Authorization: Bearer sk-1234'
curl -s -o /dev/null -X DELETE "localhost:4000/v1/responses/$ID"   -H 'x-my-new-header: hello-from-client' -H 'Authorization: Bearer sk-1234'

Before, at f60e99c (litellm_internal_staging, which already has #34531); only the create call carries the header

[relay] POST /v1/responses headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] GET /v1/responses/resp_01ab020c67f49264006a715905a54c8199862838414f8308d9 headers-seen={}
[relay] GET /v1/responses/resp_01ab020c67f49264006a715905a54c8199862838414f8308d9/input_items?limit=20&order=desc headers-seen={}
[relay] DELETE /v1/responses/resp_01ab020c67f49264006a715905a54c8199862838414f8308d9 headers-seen={}

After, at e17947b

[relay] POST /v1/responses headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] GET /v1/responses/resp_0442fca79353f58a006a7158aec824819a87f556872ab5b122 headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] GET /v1/responses/resp_0442fca79353f58a006a7158aec824819a87f556872ab5b122/input_items?limit=20&order=desc headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] DELETE /v1/responses/resp_0442fca79353f58a006a7158aec824819a87f556872ab5b122 headers-seen={'x-my-new-header': 'hello-from-client'}

Cancel and compact need a background response, so they get their own run

R=$(curl -s localhost:4000/v1/responses \
  -H 'content-type: application/json' \
  -H 'x-my-new-header: hello-from-client' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{"model":"gpt-5.6-sol","input":"write a very long essay about oceans","background":true,"store":true}')
ID=$(echo "$R" | python -c 'import sys,json;print(json.load(sys.stdin)["id"])')

curl -s -o /dev/null -X POST "localhost:4000/v1/responses/$ID/cancel" -H 'x-my-new-header: hello-from-client' -H 'Authorization: Bearer sk-1234'
curl -s -o /dev/null -X POST "localhost:4000/v1/responses/compact" \
  -H 'content-type: application/json' \
  -H 'x-my-new-header: hello-from-client' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{"model":"gpt-5.6-sol","input":"summarize: the ocean is big"}'

Before, at f60e99c

[relay] POST /v1/responses headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] POST /v1/responses/resp_0266234a53209471006a71743726f8819b961aaf2b757fe0cb/cancel headers-seen={}
[relay] POST /v1/responses/compact headers-seen={}

After, at 364deca

[relay] POST /v1/responses headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] POST /v1/responses/resp_033d6a10ca6b17cd006a717474cbf081989be43a8b5b563f86/cancel headers-seen={'x-my-new-header': 'hello-from-client'}
[relay] POST /v1/responses/compact headers-seen={'x-my-new-header': 'hello-from-client'}

Type

🐛 Bug Fix

Changes

#34531 mapped the proxy's forwarded headers kwarg onto extra_headers only in responses(), so every other responses route still dropped it. The merge is now a one-liner helper (_merge_forwarded_client_headers) applied in get_responses, delete_responses, cancel_responses, list_input_items and compact_responses as well, all reusing ResponsesAPIRequestUtils.merge_client_forwarded_headers, so precedence rules stay identical everywhere

Tests parametrize over all five routes, patch the matching httpx verb and assert both that the forwarded header reaches the provider request and that an explicit extra_headers value still wins over a differently cased client header

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/a1233d20018d4279b3a1b69094ccd33b

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR consistently forwards configured client headers through the Responses API management routes while preserving case-insensitive precedence for explicit extra_headers.

  • Adds a shared header-merging helper across response creation, retrieval, deletion, cancellation, compaction, and input-item listing.
  • Broadens the merge utility to accept mapping inputs and return a mutable dictionary.
  • Adds parameterized async tests covering forwarding and explicit-header precedence on all five management routes.
  • Updates static type-discipline budgets to reflect the changes.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
litellm/responses/main.py Centralizes forwarded-header merging and applies the effective headers consistently to all Responses API management provider calls.
litellm/responses/utils.py Broadens header inputs to mappings and normalizes merge results into dictionaries while retaining case-insensitive explicit-header precedence.
tests/test_litellm/responses/test_responses_api_request_body.py Adds mocked coverage for forwarding and precedence across retrieval, deletion, cancellation, input-item listing, and compaction.
basedpyright-code-budget.json Reduces unknown-member and unknown-variable type budgets to match the improved typing.
type-discipline-budget.json Lowers the LIT001 budget by one to reflect the type-discipline improvement.

Reviews (2): Last reviewed commit: "refactor(responses): keep merged forward..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.56522% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/responses/main.py 68.18% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_forward_client_headers_responses_management_routes (9c931cf) with litellm_internal_staging (97a59c8)

Open in CodSpeed

@SwiftWinds SwiftWinds left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your parametrized route matrix covers only get, delete, cancel, and list-input-items. Please also add compact

Also, add before=fail/after=succeed no mock real proxy live e2e proof

EDIT: did not mean to write on my personal account

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your parametrized route matrix covers only get, delete, cancel, and list-input-items. Please also add compact

Also, add before=fail/after=succeed no mock real proxy live e2e proof

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Done in 364deca: compact is in the route matrix now, and the description has a live before/after run for cancel and compact against real OpenAI (before, both reach the provider with no x-my-new-header; after, both carry it)

mateo-berri and others added 2 commits August 8, 2026 20:07
…al_staging

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Merged staging and kept the merged headers in a Final local instead of rebinding the param. @greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants