Skip to content

fix(bedrock): forward litellm_params AWS auth to batch create signing - #36449

Closed
noahnistler wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
Evernorth:litellm_bedrock_batch_cross_account_role
Closed

fix(bedrock): forward litellm_params AWS auth to batch create signing#36449
noahnistler wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
Evernorth:litellm_bedrock_batch_cross_account_role

Conversation

@noahnistler

@noahnistler noahnistler commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Cross-account Bedrock batch jobs fail with "Cross-account pass role is not allowed"
  • Deployment-level AWS auth (litellm_params) was dropped before signing the request

How it solves it:

  • Merge litellm_params into the params used for region lookup and SigV4 signing
  • The cross-account role configured on the deployment now signs the request

User Flow

Before: an admin who configured a Bedrock deployment with a cross-account batch role gets every batch job rejected by AWS

  1. They POST to http://litellm-domain/v1/files with purpose=batch, staging an input file against a Bedrock deployment whose config sets a cross-account IAM role
  2. They POST to http://litellm-domain/v1/batches with that file id, endpoint /v1/chat/completions, and completion_window 24h
  3. The response body carries an AWS error saying the cross-account pass role is not allowed, even though the deployment's role ARN is correct

After: the same batch job is accepted

  1. The admin sends the same POST http://litellm-domain/v1/files
  2. They send the same POST http://litellm-domain/v1/batches
  3. The response now returns a batch object with a job id and a validating/in-progress status instead of an error

Relevant issues

Linear ticket

Resolves LIT-4223

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Reproducing this needs a Bedrock deployment with a real cross-account IAM role (aws_role_name for the assume-role hop, aws_batch_role_arn for the job's roleArn), which I don't have credentials for. Steps below reproduce it against a live proxy for anyone who does.

Config (dev_config.yaml), pointed at a Bedrock model whose account differs from the calling identity's account:

model_list:
  - model_name: bedrock-cross-account-batch
    litellm_params:
      model: bedrock/anthropic.claude-3-haiku-20240307-v1:0
      aws_region_name: us-east-1
      aws_role_name: arn:aws:iam::<TARGET_ACCOUNT_ID>:role/<cross-account-assume-role>
      aws_batch_role_arn: arn:aws:iam::<TARGET_ACCOUNT_ID>:role/<bedrock-batch-execution-role>
      s3_bucket_name: <your-bucket>

Start the proxy against that config, then:

FILE_ID=$(curl -s -X POST http://localhost:4000/v1/files \
  -H "Authorization: Bearer sk-1234" \
  -F purpose="batch" \
  -F file="@input.jsonl" | jq -r .id)

curl -s -X POST http://localhost:4000/v1/batches \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d "{\"input_file_id\": \"$FILE_ID\", \"endpoint\": \"/v1/chat/completions\", \"completion_window\": \"24h\", \"model\": \"bedrock-cross-account-batch\"}"

Before the fix (checked out at 9de3315dad, the commit this branch is based on): the batches call fails, and the proxy log shows the AWS error body containing "Cross-account pass role is not allowed."

After the fix (this branch, 6b50fcc8a5): the same call returns a batch object with an id and a validating/in-progress status.

Type

🐛 Bug Fix

Caveats (if any)

  • The equivalent file-upload signing path (files/transformation.py) has a similar-looking gap; left untouched to keep this PR scoped to batch creation

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

transform_create_batch_request signed requests using only optional_params,
which is always {} at the create_batch call site in llm_http_handler.py.
This meant cross-account role/credentials configured via litellm_params
never reached sign_aws_request, causing Bedrock to reject the signed
request with "Cross-account pass role is not allowed" even though
aws_batch_role_arn was correctly resolved for the roleArn field.

Merge optional_params and litellm_params before signing and region
resolution so cross-account auth configured at the deployment level is
actually used to sign the CreateModelInvocationJob request.
@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR forwards deployment-level AWS authentication and region parameters into Bedrock batch request signing so configured cross-account credentials are retained.

  • Merges batch optional parameters with deployment parameters before region resolution and SigV4 signing.
  • Adds a regression test covering deployment-provided role and region forwarding.

Confidence Score: 5/5

The PR appears safe to merge, with the Bedrock batch request now using the deployment’s configured AWS authentication and region consistently.

The active batch caller supplies deployment parameters through litellm_params and an empty optional_params mapping; the merged mapping therefore restores the required signer inputs, while the pre-signed HTTP branch sends the same URL used during signing.

Important Files Changed

Filename Overview
litellm/llms/bedrock/batches/transformation.py Correctly supplies deployment AWS settings to both endpoint construction and the existing Bedrock signer without changing the HTTP transport path.
tests/test_litellm/llms/bedrock/batches/test_transformation.py Adds focused mocked coverage confirming that the configured assume-role ARN and region reach request signing.

Reviews (1): Last reviewed commit: "fix(bedrock): forward litellm_params AWS..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing Evernorth:litellm_bedrock_batch_cross_account_role (6b50fcc) with litellm_internal_staging (9de3315)

Open in CodSpeed

@mateo-berri

Copy link
Copy Markdown
Contributor

Thanks Noah, the diagnosis was right. PR #36160 merged this same litellm_params merge for batch and file signing, so closing this as superseded.

Copy link
Copy Markdown
Collaborator

@linear attach this pr BerriAI/litellm#36160to the ticket

@linear-code

linear-code Bot commented Aug 14, 2026

Copy link
Copy Markdown

Attached this pull request to LIT-5303.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants