Skip to content

feat(adapters): Add support for extra headers to more backend providers - #661

Merged
planetf1 merged 1 commit into
i-am-bee:mainfrom
planetf1:headers2
Mar 26, 2025
Merged

planetf1 merged 1 commit into
i-am-bee:mainfrom
planetf1:headers2

Conversation

@planetf1

@planetf1 planetf1 commented Mar 25, 2025 •

Copy link
Copy Markdown
Contributor

Which issue(s) does this pull-request address?

Closes: #586

Description

Adds support for extra headers to:

  • Azure OpenAI
  • Anthropic
  • Amazon Bedrock
  • Google VertexAI

In the above cases support for extra_headers is documented in the LiteLLM docs and/or found from a code scan

Changes were not made to

  • Ollama (note there is a PR open)
  • WatsonX
  • Groq
  • XAI

due to lack of support in liteLLM.

  • OpenAI adapter already has support

Testing
IN PROGRESS

  • Check modified providers still work with no extra headers
  • check extra headers sent by network traffic inspection

No test is made that the extra headers have any effect on the actual provider.

Checklist

General

Code quality checks

  • Linting passes: Python poe lint or poe lint --fix / TypeScript yarn lint or yarn lint:fix
  • Formatting is applied: Python poe format or poe format --fix / TypeScript: yarn format or yarn format:fix
  • Static type checks pass: Python poe type-check

Testing

  • Unit tests pass: Python poe test --type unit / TypeScript yarn test:unit
  • E2E tests pass: Python poe test --type e2e / TypeScript: yarn test:e2e
  • Integration tests pass: Python poe test --type integration
  • Tests are included (for bug fixes or new features)

Documentation

  • Documentation is updated
  • Embedme embeds code examples in docs. To update after edits, run: Python poe docs --type build

@planetf1

planetf1 commented Mar 25, 2025 •

Copy link
Copy Markdown
Contributor Author

Testing:

Bedrock, Vertex, Anthropic:

In addition to showing the extra_headers being set on the LiteLLM API call,
Header's show in liteLLM log as part of the curl (which makes sense) as:

curl -X POST \
https://bedrock-runtime.eu-west-2.amazonaws.com/model/meta.llama3-8b-instruct-v1:0/converse \
-H 'Content-Type: *****' -H 'sample-header: *****' -H 'X-Amz-Date: *****' -H 'Authorization: secret1 Credential=cred1/part2/eu-west-2/bedrock/aws4_request, SignedHeaders=content-type;host;sample-header;x-amz-date, Signature=0000230230********************************************' -H 'Content-Length: *****' \
-d '{"messages": [{"role": "user", "content": [{"text": "what states are part of New England?"}]}], "additionalModelRequestFields": {}, "system": [], "inferenceConfig": {"temperature": 0}}'

This is different from OpenAI (and Azure OpenAI) where there is no extra -H shown in the litellm trace, but instead the additional 'header' actually features within the body

curl -X POST \
https://randomname-location.cognitiveservices.azure.com/openai/deployments/gpt-4o-mini/ \
-H 'api_key: **obfuscated**********************************************' -H '
azure_ad_token: *****' \
-d '{'model': 'gpt-4o-mini', 'messages': [{'role': 'user', 'content': [{'type': 'text', 'text': 'what states are part of New England?'}]}], 'temperature': 0, 'stream': False, 'extra_body': {}, 'extra_headers': {'sample-header': 'abcdjfkhskdjfhskjhfksdhjfsifhskdjhfsdf'}}'

I am unable to test bedrock, vertex,anthropic, bedrock specifically in terms of the headers -- but the API is called correctly as per docs. We do also know the openai connector does work as we have a valid test system that requires headers to test against, so the fact azure behaves the same is also logical.

@planetf1
planetf1 marked this pull request as ready for review March 25, 2025 14:26
@ajbozarth ajbozarth added the python Python related functionality label Mar 25, 2025
@Tomas2D
Tomas2D requested a review from a team March 26, 2025 08:55
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1
planetf1 merged commit 4b1a337 into i-am-bee:main Mar 26, 2025
markstur pushed a commit to markstur/beeai-framework that referenced this pull request Jan 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Python related functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

adapters: Add support for extra headers to additional implementations

3 participants