fix(bedrock): emit Nova Sonic realtime session.created on connect and session.updated on session.update

[mubashir1osmani]

Relevant issues

Linear ticket

Resolves LIT-4655

fixes a failing e2e test - https://github.com/BerriAI/litellm/blob/litellm_internal_staging/tests/e2e/llm_translation/realtime/test_nova_sonic_realtime_e2e.py

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Live, against real AWS Bedrock bedrock/amazon.nova-sonic-v1:0 (mode: realtime), no mocks. The client follows the OpenAI Realtime protocol: it waits for session.created before sending anything.

Config (nova_realtime_only.yaml):

model_list:
  - model_name: "bedrock-sonic"
    litellm_params:
      model: bedrock/amazon.nova-sonic-v1:0
      aws_region_name: us-east-1
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
    model_info:
      mode: realtime
general_settings:
  master_key: "sk-1234"

Launch (needs aws-sdk-bedrock-runtime installed, i.e. pip install 'litellm[bedrock-realtime]'):

set -a; source .env; set +a   # AWS creds
python litellm/proxy/proxy_cli.py --config nova_realtime_only.yaml --detailed_debug --port 30751

Probe client (waits for session.created first, then session.update -> conversation.item.create -> response.create, logs every event type):

import asyncio, json, websockets
URL = "ws://localhost:30751/v1/realtime?model=bedrock-sonic"
async def main():
    async with websockets.connect(URL, additional_headers={"Authorization": "Bearer sk-1234"}) as ws:
        first = json.loads(await asyncio.wait_for(ws.recv(), timeout=15))
        print("FIRST EVENT ON CONNECT:", first.get("type"))
        await ws.send(json.dumps({"type":"session.update","session":{"instructions":"You are helpful.","modalities":["text"]}}))
        await ws.send(json.dumps({"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Say hello in one short sentence."}]}}))
        await ws.send(json.dumps({"type":"response.create"}))
        while True:
            ev = json.loads(await asyncio.wait_for(ws.recv(), timeout=45))
            if ev.get("type") not in ("response.audio.delta","response.text.delta"):
                print("event:", ev.get("type"))
            if ev.get("type") == "response.done":
                print("GOT response.done"); break
asyncio.run(main())

Before (shipping code, base 7015bd2ea1)

The server never emits session.created on connect, so the probe blocks forever on its first recv():

(client blocks on await ws.recv(); killed after 45s with zero events)

Proxy side showed the socket was accepted but nothing was ever sent back:

INFO:     "WebSocket /v1/realtime?model=bedrock-sonic" [accepted]
INFO:     connection open
(no session.created, no further frames)

After (this PR, 6d7d03d194)

FIRST EVENT ON CONNECT: session.created
event: session.updated
event: response.created
event: response.output_item.added
event: response.content_part.added
event: response.text.done
event: response.content_part.done
event: response.output_item.done
event: response.created
[... response.audio.delta x80 elided ...]
event: response.audio.done
event: response.content_part.done
event: response.output_item.done
event: response.done
GOT response.done

session.created arrives unprompted on connect, session.updated acks the client's session.update, and the full realtime turn completes through response.done.

Type

🐛 Bug Fix

Changes

PR #31924 (LIT-2239) fixed a different Nova Sonic realtime hang, the response.create no-op and the missing response.done, but left a second protocol gap: the server never emits session.created unprompted on connect, and never emits session.updated in response to the client's session.update. Per the OpenAI Realtime protocol the server must send session.created immediately on connect, and conformant clients (and the real SDKs) wait for it before sending session.update. So they deadlock; the client waits on the server, the client-to-Bedrock forwarding loop waits on the client, and Bedrock waits on litellm. PR #31924's own QA client sidestepped this by sending session.update without waiting for session.created, so it was never caught. The only code that produced session.created was gated on a Bedrock output event containing sessionStart, which Nova Sonic v1 never emits (it is an input-only event)

The handler now emits a synthesized session.created to the client immediately after the bidirectional stream is established, independent of client input, and emits session.updated when the client sends session.update. The event bodies are built by two small BedrockRealtimeConfig helpers (session_created_event, session_updated_event) that share one _session_object; the existing reactive sessionStart mapping now delegates to the same builder, so its tested behavior is unchanged

The realtime handler lazily imports aws-sdk-bedrock-runtime (plus smithy-aws-core), the experimental AWS SDK providing InvokeModelWithBidirectionalStream, which boto3 cannot do. That package was not declared anywhere in pyproject.toml, so a stock install raised ImportError and closed the socket with 1011 before the model could run. This adds it as a pinned bedrock-realtime optional extra. Wiring the extra into the gateway image is tracked separately

Review follow-ups: the session.updated ack now reflects the client's requested modalities rather than a fixed ["text", "audio"], so a text-only session.update is acknowledged as text-only. The reactive Bedrock sessionStart mapping no longer forwards a second session.created, so session.created is emitted exactly once (on connect) even if a compatible stream later emits sessionStart

uv.lock is regenerated for the new extra. aws-sdk-bedrock-runtime requires Python >=3.12 while litellm supports >=3.10, so the dependency carries a python_version >= '3.12' marker (same pattern as other version-gated extras); on 3.10/3.11 the extra is empty and the lazy import behaves as before

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR completes the Nova Sonic realtime session lifecycle. The main changes are:

• Emits session.created after establishing the Bedrock stream
• Emits session.updated after forwarding a client session update
• Prevents backend sessionStart events from creating duplicate lifecycle events
• Adds the Bedrock realtime SDK as an optional dependency
• Adds focused lifecycle and transformation tests

Confidence Score: 5/5

This looks safe to merge.

• The duplicate creation event is now suppressed.
• Session updates now receive an acknowledgement with the requested modalities.
• The frozen dependency graph now includes the optional Bedrock realtime SDK.
• No blocking issues remain in the changed code.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/realtime/handler.py	Adds connection-time session creation and client update acknowledgements.
litellm/llms/bedrock/realtime/transformation.py	Centralizes session event construction and suppresses duplicate creation events.
pyproject.toml	Declares the Bedrock realtime SDK as a Python 3.12+ optional dependency.
uv.lock	Adds the Bedrock realtime dependency and its resolved package graph.
tests/test_litellm/llms/bedrock/realtime/test_bedrock_realtime_handler.py	Adds tests for connection and session-update lifecycle events.
tests/test_litellm/llms/bedrock/realtime/test_bedrock_realtime_transformation.py	Adds session event shape tests and verifies duplicate suppression.

_{Reviews (2): Last reviewed commit: "fix(bedrock): emit realtime session.crea..." | Re-trigger Greptile}

[mubashir1osmani]

@greptileai pushed a follow-up addressing both findings: the session.updated ack now reflects the client's requested modalities, and the reactive Bedrock sessionStart no longer forwards a duplicate session.created (single-source on connect). Also regenerated uv.lock for the new bedrock-realtime extra

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_nova_realtime_session_created (78cf391) with litellm_internal_staging (a780d4e)}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mubashir1osmani
❌ Mubashir Osmani

---

Mubashir Osmani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}