fix(gateway): install bedrock-realtime extra for Nova Sonic realtime

[mubashir1osmani]

TLDR

Problem this solves:

• Bedrock Nova Sonic realtime fails on the gateway: Missing aws_sdk_bedrock_runtime
• the gateway image never installed the bedrock-realtime extra

How it solves it:

• add --extra bedrock-realtime to both uv sync stages in gateway/Dockerfile

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Captured at 14b81bb. This is an image-dependency fix, so the proof is the missing package on the live gateway plus the lock already carrying it; the end-to-end Nova Sonic pass follows the image rebuild + deploy.

Gateway log (the realtime request that failed):

litellm.router ... Error occurred while trying to do fallbacks - Missing aws_sdk_bedrock_runtime.
Install with: pip install aws-sdk-bedrock-runtime

Confirmed absent on the running gateway:

$ kubectl exec -n litellm <gateway-pod> -- python -c "import importlib.util as u; print(u.find_spec('aws_sdk_bedrock_runtime') is not None)"
False

Already locked, so uv sync --frozen --extra bedrock-realtime resolves without a relock:

$ grep -n 'name = "aws-sdk-bedrock-runtime"' uv.lock
534:name = "aws-sdk-bedrock-runtime"

Type

🐛 Bug Fix

Changes

gateway/Dockerfile installed --extra proxy, proxy-runtime, extra_proxy, and semantic-router, but not bedrock-realtime. Bedrock Nova Sonic speech-to-speech uses the InvokeModelWithBidirectionalStream API, which boto3 cannot do; litellm drives it through aws-sdk-bedrock-runtime (with its transitive smithy-* deps), imported lazily in the realtime handler. Without the extra the import fails and every Nova Sonic realtime request errors at startup. Added --extra bedrock-realtime to both uv sync stages (builder and final). The extra is already in uv.lock, and its aws-sdk-bedrock-runtime>=0.7.0,<0.8.0; python_version >= '3.12' marker matches the python3.13 image.

Scope note: the monolith proxy images (Dockerfile, docker/Dockerfile.non_root, docker/Dockerfile.database) serve /v1/realtime too and have the same gap; this PR fixes only the componentized gateway that's deployed and erroring. Happy to extend to those in this PR or a follow-up if preferred. The backend and migrations images do not serve realtime, so they are intentionally untouched.

QA runbook

• Build the gateway image from this branch and confirm the package is present: docker run --rm --entrypoint python <image> -c "import aws_sdk_bedrock_runtime; print('ok')"
• Deploy it to stage and re-run tests/e2e/llm_translation/realtime/test_realtime_bedrock_e2e.py::TestNovaSonicRealtime::test_nova_sonic_response_create_completes; expect it to pass rather than error with Missing aws_sdk_bedrock_runtime
• Sanity check: a non-realtime Bedrock call (e.g. bedrock_converse) still works, confirming the extra did not perturb the existing resolve

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

Adds the Bedrock realtime dependency extra to the gateway image installation.

• Includes bedrock-realtime in both frozen uv sync stages.
• Ensures the final gateway virtual environment contains the dependency needed by Nova Sonic realtime.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename	Overview
gateway/Dockerfile	Adds the existing locked bedrock-realtime extra consistently to the dependency and project installation stages.

_{Reviews (2): Last reviewed commit: "fix(gateway): install bedrock-realtime e..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_gateway_bedrock_realtime_extra (14b81bb) with litellm_internal_staging (43e7b96)}