fix(bedrock): forward each Nova Sonic assistant sentence once over the realtime API

[devin-ai-integration]

TLDR

Problem this solves:

• Nova 2 Sonic sends each assistant sentence twice: a SPECULATIVE preview, then a FINAL transcript
• The realtime bridge forwarded both, so voice apps rendered every bubble twice
• The FINAL copies arrive after response.done, as extra responses that never close

How it solves it:

• ASSISTANT text blocks tagged generationStage FINAL are dropped whole
• Their textOutput and contentEnd emit nothing and open no response
• SPECULATIVE text still streams next to the audio inside the one response

Decision, which text stage to forward: the SPECULATIVE block, since it streams in step with the audio and is the one AWS's own Nova Sonic sample client displays. The alternative, forwarding FINAL instead, would give the exact spoken words under barge-in but arrives 2 to 13 seconds after the audio turn ended, so the text would trail the voice and the response would have to stay open past its audio

Decision, the barge-in marker: when the user talks over the assistant, Nova Sonic sends { "interrupted" : true } as a FINAL text block, so it is dropped with the rest of that stage. Before this PR it reached the client as a response.text.done, a JSON string in the transcript; clients learn about barge-in from input_audio_buffer.speech_started, which the bridge still emits at the same instant. The alternative, forwarding that one block, keeps that string as an assistant sentence

User Flow

Before: a voice chat app on the gateway hears Nova 2 Sonic's answer once but renders every sentence of it twice, the second copy landing seconds after the voice stopped

1. The app opens ws://litellm-domain/v1/realtime?model=nova-2-sonic with its key and sends session.update with modalities text and audio
2. It streams the user's spoken question as input_audio_buffer.append chunks and keeps streaming silence like a live mic
3. response.created arrives, then each sentence as response.text.delta and response.text.done beside its response.audio.delta chunks, then response.done with the turn's usage
4. 2 to 13 seconds later a second response.created repeats the first sentence and closes with a zero-usage response.done, then a third response.created repeats the rest and never gets a response.done
5. The chat UI shows every assistant bubble twice and the app sits waiting on a response that never ends

After: the same app renders each sentence once, inside the one response that carries the audio

1. The app opens ws://litellm-domain/v1/realtime?model=nova-2-sonic with its key and sends session.update with modalities text and audio
2. It streams the user's spoken question as input_audio_buffer.append chunks and keeps streaming silence like a live mic
3. response.created arrives, then each sentence as response.text.delta and response.text.done beside its response.audio.delta chunks, then response.done with the turn's usage
4. Nothing else arrives for the turn: no second response.created, no repeated sentences, no dangling response
5. The chat UI shows each assistant bubble once

Linear ticket

Resolves LIT-8997

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Last updated: 6451016. QA and /live-pr-risk ran on 07107e5, the last commit with a product diff; 48cd6d4 and 6451016 only retype, tag, and extend the tests. /audit round 1 ran on 6451016 (same product code as 07107e5) against the merge base e98bbc2

Shared setup: one proxy process per leg, booted from the named commit with --num_workers 2 (two uvicorn workers) on a random free port, a Postgres DB, store_model_in_db: true, AWS keys for us-east-1 in its env, and this config. Both legs ran concurrently with the same client and the same flow

model_list:
  - model_name: nova-2-sonic
    litellm_params:
      custom_llm_provider: bedrock
      aws_region_name: us-east-1
      model: amazon.nova-2-sonic-v1:0

The client is a 118-line websocket script (nova_sonic_realtime_client.py) that does what a voice chat app does: session.update with modalities text and audio, then a spoken "what is the weather in paris?" (the repo's tests/e2e/llm_translation/realtime/fixtures/weather_question_24k.wav) as 100 ms PCM16 16 kHz input_audio_buffer.append chunks followed by silence until 15 s pass with no event. It prints every non-audio event with its arrival time, then the counts. Real Nova 2 Sonic calls, real spend

Before (e98bbc2, the merge base)

One spoken turn over ws://127.0.0.1:31137/v1/realtime?model=nova-2-sonic

1. python nova_sonic_realtime_client.py 31137 "$LITELLM_MASTER_KEY" nova-2-sonic events.jsonl
2. Observed timeline (audio deltas omitted)

[  2.918] conversation.item.input_audio_transcription.completed 'what is the weather in paris?'
[  3.542] response.created    response_id=resp_c3255653
[  3.542] response.text.done  "I don't have real-time data, so I can't provide the current weather in Paris."
[  4.511] response.text.done  ' You can check a reliable weather website or app for up-to-date information.'
[  5.446] response.text.done  ' They often give detailed forecasts and conditions for any location.'
[  6.296] response.done       response_id=resp_c3255653 usage={'input_tokens': 551, 'output_tokens': 386, ...}
[  8.484] response.created    response_id=resp_81d90332
[  8.484] response.text.done  "I don't have real-time data, so I can't provide the current weather in Paris."
[  8.484] response.done       response_id=resp_81d90332 usage={'input_tokens': 0, 'output_tokens': 0, 'total_tokens': 0, ...}
[ 13.421] response.created    response_id=resp_63620fe1
[ 13.421] response.text.done  ' You can check a reliable weather website or app for up-to-date information.'
[ 17.881] response.text.done  ' They often give detailed forecasts and conditions for any location.'
(no response.done for resp_63620fe1; the client's 15 s idle timer ends the session)

=== sentences delivered more than once: all three, 2x each
response.created: 3 | response.text.done: 6 | response.done: 2

The e2e test in this PR against that proxy

1. LITELLM_PROXY_URL=http://127.0.0.1:31137 uv run --group e2e-dev pytest tests/e2e/llm_translation/realtime/test_realtime_bedrock_duplicate_text_e2e.py -v
2. Observed

FAILED tests/e2e/llm_translation/realtime/test_realtime_bedrock_duplicate_text_e2e.py::TestNovaSonicAssistantText::test_nova_sonic_voice_turn_shows_each_sentence_once
AssertionError: assistant text delivered more than once: {"I don't have real-time data, so I can't provide the current weather in Paris.": 2, 'You can check a reliable weather website or app for up-to-date information.': 2, 'They will give you the most accurate forecast right now.': 2}
1 failed in 46.68s

After (07107e5, the last commit with a product diff)

One spoken turn over ws://127.0.0.1:29927/v1/realtime?model=nova-2-sonic

1. python nova_sonic_realtime_client.py 29927 "$LITELLM_MASTER_KEY" nova-2-sonic events.jsonl
2. Observed timeline (audio deltas omitted)

[  2.913] conversation.item.input_audio_transcription.completed 'what is the weather in paris?'
[  3.498] response.created    response_id=resp_8d806a58
[  3.498] response.text.done  "I don't have real-time capabilities to provide current weather information."
[  4.376] response.text.done  ' You can check the latest weather for Paris using a weather website or app for up-to-date details.'
[  5.549] response.text.done  ' They will give you accurate forecasts and conditions.'
[  6.227] response.done       response_id=resp_8d806a58 usage={'input_tokens': 551, 'output_tokens': 382, ...}
(nothing else arrives; the client's 15 s idle timer ends the session)

=== sentences delivered more than once: none
response.created: 1 | response.text.done: 3 | response.done: 1

The e2e test in this PR against that proxy

1. LITELLM_PROXY_URL=http://127.0.0.1:29927 uv run --group e2e-dev pytest tests/e2e/llm_translation/realtime/test_realtime_bedrock_duplicate_text_e2e.py -v
2. Observed

tests/e2e/llm_translation/realtime/test_realtime_bedrock_duplicate_text_e2e.py::TestNovaSonicAssistantText::test_nova_sonic_voice_turn_shows_each_sentence_once PASSED
1 passed in 36.29s

Observations from the run

• Before: the third response never gets a response.done; this PR removes it
• Before: the second response.done reports all-zero usage; this PR removes it
• The replayed text arrives 2 to 12 s after the audio ends; this PR drops it
• Audio delta counts differ per turn (156 vs 153); model variance, not this PR

/live-pr-risk, 07107e5 against the merge base e98bbc2

Verdict: no dependent path breaks; one backward-incompatible observable, decided in the TLDR. Same two-worker proxies as the QA above (before on port 31137 from the merge-base worktree, after on port 29927 from this branch), the same websocket client, both scenarios run on both legs at the same time. The after proxy's debug log shows 19 Dropping FINAL ASSISTANT text block lines over these runs, the before log none

Breaking

None. Per spoken session, counted from the client's event stream:

Scenario	Leg	response.created	response.done	text.done	Sentences twice	Never closed
Barge-in at 0.8 s	before	3	2	8	3	1
Barge-in at 0.8 s	after	1	1	4	0	0
Second turn 3 s after response.done	before	5	4	12	5	1
Second turn 3 s after response.done	after	2	2	6	0	0

Barge-in: the client speaks the question again 0.8 s after the first response.audio.delta. Both legs emit input_audio_buffer.speech_started at the moment the mic audio resumes, and the answer continues inside the same response id on both. Second turn: the client speaks again 3 s after the first response.done; before, the first turn's FINAL blocks were still arriving and opened phantom responses, one of which swallowed the second turn's first sentence into its own response id

Spend rows (GET /spend/logs, one row per session on both legs): completion_tokens equals the sum of the response.done usage the client saw. Before: 605 and 722 (the phantom response.done events carried zero usage and added nothing). After: 495 and 702. Same rule, same row shape, same call_type

Backward incompatible

• The barge-in marker. Nova Sonic sends { "interrupted" : true } as a FINAL text block whose contentEnd carries stopReason: INTERRUPTED (both proxies' debug logs show that block). Before: forwarded as a response.text.done with that JSON string as the text. After: dropped with the FINAL stage, while input_audio_buffer.speech_started still reaches the client at the same instant. Decision recorded in the TLDR, shipped with this PR's approval; the unit test test_barge_in_marker_in_final_block_is_dropped_and_the_next_answer_still_flows pins it

Regression risk

• An interrupted response still ends with one ordinary response.done on both legs, never a cancelled status; pre-existing, both legs identical, out of scope
• Under barge-in the SPECULATIVE text can name a sentence the voice never finished (Caveats); the FINAL block was the one that omitted it, and both legs show the same SPECULATIVE sentences

Dependency graph

• transform_content_start_event, transform_text_output_event, transform_content_end_event: one implementer, one caller (transform_realtime_response), no subclass, no string dispatch; verified live on both scenarios and the QA turn
• _parse_generation_stage: already used by the USER transcript path, whose unit tests are unchanged and pass; verified live (conversation.item.input_audio_transcription.completed once per spoken question on both legs)
• _assistant_final_text_active: per-session attribute on BedrockRealtimeConfig, which the handler instantiates per websocket session; verified live across two turns in one session and across two workers
• Event consumers: the handler's logged-event filter and the realtime cost calculator (sum of response.done usage); verified live through the spend rows above
• litellm_core_utils/realtime_streaming.py: the path every event above crossed; verified live
• Docs: no page names Nova Sonic generation stages; nothing to update
• Swept litellm/, enterprise/, tests/, ui/ for the method names, the attribute, the stage strings, and the interrupted marker: no other reference of any kind

Not verified

• An outbound forwarding recorder: not applicable here. The diff touches no outbound code, the only frames the proxy sends Bedrock are the client's own audio chunks (identical on both legs by construction), and the Bedrock bidirectional stream is a SigV4-signed HTTP/2 stream through the AWS CRT client, which a forwarding recorder cannot sit in front of
• amazon.nova-sonic-v1:0, the first-generation model: not driven (Caveats)
• Merge-ref drift: origin/main is two commits past the merge base (8adb8f3, an Azure deprecation date; 62dee3d, Claude Code gateway refresh tokens). The merge is clean and neither touches a realtime module, so the merged tree's realtime code is byte-identical to this tip's; no merged-tree rerun

/audit round 1 of 3, 6451016 against the merge base e98bbc2

Verdict PASS: every inventory row verified, no row deferred, no mock inside the proxy. Head 6451016 (this tip, product diff unchanged since 07107e5), merge base e98bbc2 (git merge-base origin/main 64510165e61), each booted from its own worktree with --num_workers 2 on its own port against its own Postgres database, Redis and scripted upstream, every worktree's .env moved out before the boot (zero Environment variables loaded from lines in all five proxy logs, the two chaos restarts included)

Every Nova Sonic row is a live cell, run once per leg against the real amazon.nova-2-sonic-v1:0 in us-east-1, because no deterministic tests/integration/ cell can reach this path today: the realtime handler talks to Bedrock through aws_sdk_bedrock_runtime's CRT client, which negotiates HTTP/2 only through TLS ALPN against the system trust store with no CA bundle or verify hook, and over plaintext HTTP/1.1 smithy buffers the whole duplex body before sending it, so a scripted local peer can never carry the bidirectional stream. That testability gap is tracked internally; no deterministic test was committed by this audit, and the live cells are not committed either. Artifacts per row under the run's scratch directory, audit/live/<sha>/: <row>_events.jsonl (every event the client received), <row>_summary.json (checks, per-response usage, close code), <row>_bedrock_log.txt (the proxy's Received from Bedrock lines with the vendor's completionId), <row>_spend_rows.json (the session's LiteLLM_SpendLogs rows). Runner: audit/live/run_live.py (N1-N9) and audit/live/chaos_live.py (N12)

Row	Surface and client	Scenario	Base e98bbc2	Head 6451016	Ids (head)	Artifacts	Result	Last ran at
N1	/v1/realtime, raw websockets, nova-2-sonic	audio sent before any session.update	error event ValidationException: The first event must be a SessionStart event. (the runner stops reading at the error event), no spend row, next session served	error event ValidationException: The first event must be a SessionStart event. (the runner stops reading at the error event), no spend row, next session served		<run scratch>/audit/live/<sha>/N1_*	PASS	6451016
N2	/v1/realtime, OpenAI SDK async realtime.connect	session.update, one spoken turn	created/done 3/2, text.done 6, repeated x2, x2, x2, usage in/out 551/366, spend row 775dfce3	created/done 1/1, text.done 3, repeated none, usage in/out 551/363, spend row ca716978	resp_19b339be-71a7-4762-86b6-a5da2f04de9e; vendor completionId ad15616b	<run scratch>/audit/live/<sha>/N2_*	PASS	6451016
N3	/v1/realtime, OpenAI SDK sync realtime.connect	session.update, one spoken turn	created/done 3/2, text.done 6, repeated x2, x2, x2, usage in/out 551/382, spend row fe064c9b	created/done 1/1, text.done 3, repeated none, usage in/out 551/364, spend row f6f25745	resp_9822ffc7-6034-419a-8733-107c5a9e4f96; vendor completionId 281db46f	<run scratch>/audit/live/<sha>/N3_*	PASS	6451016
N4	/v1/realtime, raw websockets	two spoken turns, second 3 s after response.done	created/done 5/4, text.done 12, repeated x2, x2, x2, x2, x2, marker text, usage in/out 618/702, spend row 26f7c89e	created/done 2/2, text.done 6, repeated none, usage in/out 617/687, spend row 51f5b11a	resp_72e0dce5-8279-4f7e-84cf-cfb8b3b79d6f,resp_37d3cc11-f948-48c2-ac3f-0863130a6006; vendor completionId b280afd1	<run scratch>/audit/live/<sha>/N4_*	PASS	6451016
N5	/v1/realtime, raw websockets	barge-in 0.8 s after the first response.audio.delta	created/done 3/2, text.done 8, repeated x2, x2, x2, marker text, usage in/out 620/603, spend row 7866cadd	created/done 1/1, text.done 4, repeated none, usage in/out 618/589, spend row f13c3a7a	resp_968b8339-5e19-4b0a-b950-2b9e7c417fa6; vendor completionId 6e6bb2bc	<run scratch>/audit/live/<sha>/N5_*	PASS	6451016
N6	/v1/realtime, raw websockets	text item turn, conversation.item.create + response.create	created/done 1/1, text.done 3, repeated none, usage in/out 582/411, spend row dc871881	created/done 1/1, text.done 3, repeated none, usage in/out 582/411, spend row fcc1f81c	resp_bf9821ab-b7ea-4633-9c08-f8524767d077; vendor completionId df2bbc61	<run scratch>/audit/live/<sha>/N6_*	PASS	6451016
N7	/v1/realtime, raw websockets, nova-2-sonic-unknown	unknown model id amazon.nova-2-sonic-v9:0, then a nova-2-sonic session	error event litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are tryin (the runner stops reading at the error event), no spend row, next session served	error event litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are tryin (the runner stops reading at the error event), no spend row, next session served		<run scratch>/audit/live/<sha>/N7_*	PASS	6451016
N8	/v1/realtime, raw websockets, nova-2-sonic-badcreds	bogus static AWS keys, then a nova-2-sonic session	error event Unknown error for operation com.amazonaws.bedrockruntime#InvokeModelWithBidirectionalStrea (the runner stops reading at the error event), no spend row, next session served	error event Unknown error for operation com.amazonaws.bedrockruntime#InvokeModelWithBidirectionalStrea (the runner stops reading at the error event), no spend row, next session served		<run scratch>/audit/live/<sha>/N8_*	PASS	6451016
N9	/v1/realtime, raw websockets x4 concurrent	one spoken turn each, lsof on both workers during the burst	created/done per session [(3, 2), (3, 2), (3, 2), (3, 2)], repeated sentences [3, 3, 3, 3], 4 spend rows, workers with a Bedrock socket during the burst: 2	created/done per session [(1, 1), (1, 1), (1, 1), (1, 1)], repeated sentences [0, 0, 0, 0], 4 spend rows, workers with a Bedrock socket during the burst: 2	vendor completionIds 0ee0b5dd,37543320,a52d2683,db13ff61	<run scratch>/audit/live/<sha>/N9_*	PASS	6451016
N12	chaos, the leg's 2-worker proxy	SIGKILL the worker holding a session; new session; SIGTERM restart; new session	socket closed 0.02 s after SIGKILL of worker 82326 (code 1006); next session served, repeats=3; restart ready in 61.2 s, next session served, repeats=3	socket closed 0.02 s after SIGKILL of worker 82319 (code 1006); next session served, repeats=0; restart ready in 48.2 s, next session served, repeats=0		<run scratch>/audit/live/<sha>/N12_summary.json, N12-after-*	PASS	6451016

Reading the matrix: on every audio turn the merge base delivers each sentence twice, opens one response more than it closes (created/done 3/2, 5/4) and on N4 and N5 forwards the barge-in marker { "interrupted" : true } as text; the head delivers each sentence once, closes every response it opens and forwards no marker. N6 (a text item turn) repeats nothing on the base either, since the FINAL block Nova sent on that path carried no text (the inventory allowed either outcome on the base). On every served session, on both legs, the spend row's completion_tokens equals the sum of the response.done output_tokens the client saw, prompt_tokens the sum of input_tokens, and spend is above zero (the spend_row check). N1, N7 and N8 are inherited failure shapes identical on both legs and outside this diff (Caveats). N10 (stage-less blocks, the USER transcript and audio, forwarded) is the transcription_completed and audio_present check on N2-N6, N9 and every follow-up session; N11 (one spend row per session with matching usage) is the spend_row check on the same rows

N9: four concurrent sessions, each one response with no repeat on the head, four spend rows with matching usage, and lsof during the burst showed both uvicorn workers holding a Bedrock socket on both legs. N12: SIGKILL of the worker holding a session closed the client's socket 0.02 s later with code 1006 on both legs, uvicorn respawned the worker and the next session was served (no repeat on the head, repeats on the base); a SIGTERM restart of the whole proxy had the listener gone in 0.1 s and ready again in 48.2 s (head) and 61.2 s (base), next session served

Controls, the realtime integration cells, run once on the head on a third rig (stock tests/integration/proxy_config.yaml, scripted upstream, uv run --no-sync python tests/integration/run.py providers <13 node ids> --seed 20261006 --order-seed 20261006, INTEGRATION_PROXY_READY_SECONDS=240, serial as CircleCI runs the group): 13 passed in 121 s, lane log at audit/results/ctl/lane.log, junit at audit/results/ctl/junit.xml

Control	Node id	Head (once, ctl rig)
C1	tests/integration/providers/test_openai_realtime_transcription_url.py::test_openai_sdk_async_transcription_session_reaches_a_custom_api_base	PASSED
C2	tests/integration/providers/test_openai_realtime_transcription_url.py::test_openai_sdk_sync_transcription_session_reaches_a_custom_api_base	PASSED
C3	tests/integration/providers/test_openai_realtime_transcription_url.py::test_conversation_session_forwards_only_model_and_bills_the_scripted_usage	PASSED
C4	tests/integration/providers/test_openai_realtime_transcription_url.py::test_upstream_outage_closes_every_open_transcription_session_and_recovers	PASSED
C5	tests/integration/providers/test_openai_realtime_transcription_url.py::test_repeated_conversation_sessions_write_one_spend_row_each	PASSED
C6	tests/integration/providers/test_openai_realtime_transcription_url.py::test_concurrent_transcription_sessions_each_reach_the_upstream_with_model_and_intent	PASSED
C7	tests/integration/providers/test_openai_realtime_transcription_url.py::test_worker_kill_then_proxy_restart_keep_transcription_sessions_serving	PASSED
C8	tests/integration/providers/test_bedrock_gpt_responses_native_wire.py::test_async_openai_sdk_responses_stream_is_served_by_the_native_responses_route	PASSED
C9	tests/integration/messages_endpoint/providers/bedrock/test_bedrock_messages_gpt_chat_completions_wire.py::test_async_anthropic_sdk_disabled_thinking_reaches_native_chat_completions_as_effort_none	PASSED
C10	tests/integration/messages_endpoint/providers/bedrock/test_bedrock_messages_gpt_chat_completions_wire.py::test_anthropic_sdk_stream_with_thinking_budget_is_served_by_native_chat_completions	PASSED
C11	tests/integration/providers/test_bedrock_runtime_chat_completions_wire.py::test_openai_sdk_reasoning_request_is_served_by_native_chat_completions	PASSED
C12	tests/integration/providers/test_bedrock_runtime_chat_completions_wire.py::test_sigv4_deployment_signs_the_native_request	PASSED
C13	tests/integration/providers/test_bedrock_runtime_chat_completions_wire.py::test_async_openai_sdk_stream_keeps_the_upstream_id_and_usage	PASSED

Run notes: the first controls run failed 10 of 13 on the rig, not the product (the owned Postgres cluster was initialized without -E UTF8, so its databases were SQL_ASCII and psycopg returned bytes for text columns, which the harness's spend-row model rejects); the controls database was recreated as UTF8, the proxy rebooted and the lane rerun above passed 13 of 13. The first live attempt hit the same bytes problem inside the runner after two rows and was discarded (attempt1_runnerbug/); the matrix above is the second attempt, run end to end on both legs. Both live proxies were restarted by the N12 chaos step and served the rows that followed

Type

🐛 Bug Fix

Caveats (if any)

Low

• Under barge-in the SPECULATIVE text can name a sentence the voice never finished; the dropped FINAL block was the one that omitted it. Left as is: forwarding FINAL instead trails the audio by seconds and keeps the response open past it, and OpenAI Realtime has no event that retracts text already delivered
• Barge-in: the { "interrupted" : true } text block no longer reaches the client (decision in the TLDR); input_audio_buffer.speech_started still does at the same instant. Left as is: forwarding that one FINAL block puts a JSON string in the transcript, which is what every client saw before this PR
• Tested on amazon.nova-2-sonic-v1:0 only; amazon.nova-sonic-v1:0 tags its text blocks the same way per AWS's sample client, so it gets the same drop. Left as is: the report names the 2 Sonic model, and the drop keys on the stage tag, not the model id
• The e2e test spends a live Nova 2 Sonic turn per run and needs AWS credentials in the proxy's env. Left as is: a scripted Bedrock stream would not prove the provider's block order, which is the whole bug
• Inherited, identical on the merge base and outside this diff (audit N1): a client that streams audio before sending session.update gets Bedrock's own ValidationException: The first event must be a SessionStart event. as an error event and the session ends, because the Bedrock realtime handler never sends the default session configuration the way the generic realtime path does. Left as is: tracked internally as its own fix, not part of the duplicate-text change
• Inherited, identical on the merge base and outside this diff (audit N7): a deployment with custom_llm_provider: bedrock and a model id Bedrock does not know (amazon.nova-2-sonic-v9:0) fails the session with litellm.BadRequestError: LLM Provider NOT provided instead of a message naming the unknown model. Left as is: tracked internally as its own fix

QA runbook

• tests/e2e/llm_translation/realtime/test_realtime_bedrock_duplicate_text_e2e.py::TestNovaSonicAssistantText::test_nova_sonic_voice_turn_shows_each_sentence_once - one spoken turn to Nova 2 Sonic delivers each assistant sentence in exactly one response.text.done
  • Proxy env has AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION=us-east-1 and store_model_in_db: true; POST /model/new with the master key, model_name of your choice, litellm_params.model: bedrock/amazon.nova-2-sonic-v1:0, aws_region_name: os.environ/AWS_REGION, and model_info.mode: realtime
  • Open ws://localhost:4000/v1/realtime?model= with a key, wait for session.created, send {"type":"session.update","session":{"modalities":["text","audio"],"instructions":"You are a friendly assistant. Answer in two or three short sentences."}}, wait for session.updated
  • Send tests/e2e/llm_translation/realtime/fixtures/weather_question_24k.wav resampled to mono PCM16 16 kHz as base64 input_audio_buffer.append chunks of 100 ms, then keep sending 100 ms of silence every 100 ms until no event has arrived for 15 s
  • Expect a response.done, and collect the text of every response.text.done: at least one sentence, and no sentence more than once
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 07107e5 passes /live-pr-risk

• 6451016 passes /audit (round 1 of 3, PASS, no unverified rows)

---

Note

Medium Risk
Changes Bedrock realtime event translation with a backward-incompatible drop of FINAL assistant text (including the barge-in JSON marker); scope is Nova Sonic’s generationStage pattern but affects all consumers of /v1/realtime for that model.

Overview
Fixes duplicate assistant chat bubbles on Bedrock Nova Sonic realtime by not forwarding ASSISTANT TEXT blocks whose contentStart carries generationStage: FINAL. Nova sends each sentence twice (SPECULATIVE with audio, then FINAL after the turn); the bridge now keeps only SPECULATIVE text inside the single response.created / response.done lifecycle and suppresses matching textOutput / contentEnd via _assistant_final_text_active.

Tests: unit scenarios for FINAL replay, post–END_TURN FINAL, barge-in { "interrupted" : true } FINAL drops, and a live e2e that streams mic audio through new RealtimeSession.stream_and_collect and asserts each response.text.done sentence appears once.

Observable change: clients no longer receive the FINAL-stage barge-in JSON as response.text.done (speech boundaries unchanged).

^{Reviewed by Cursor Bugbot for commit 6451016. Bugbot is set up for automated code reviews on this repo. Configure here.}

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Fixes duplicate text in realtime streaming responses.

The PR appears safe to merge; the latest change adds a focused interruption test without changing product behavior

What we checked:

• Next answer still arrives: The matching contentEnd clears the drop flag. The new test requires the following speculative sentence to appear beside the first sentence

Summary

This PR drops Nova Sonic's FINAL assistant text blocks so spoken sentences reach realtime clients once, alongside the audio

• Since the previous review, only an interrupted-turn unit test was added. It checks that the interruption marker and repeated FINAL text disappear while the next answer still reaches the client
• The previous helper-type, metadata, and recorded-step concerns have been addressed. All supplied threads are resolved and unnumbered, so none require a numbered previous-finding entry
• devin-ai-integration[bot] explicitly accepts dropping the FINAL interruption marker because clients still receive input_audio_buffer.speech_started. They also accept that speculative text may include words the voice never finishes during an interruption
• No new actionable issue was found. This review used code inspection, not a new live-provider run

_{Reviews (3) · Last reviewed commit: "test(bedrock): pin that the Nova Sonic b..." · Reviewed by Greptile}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_nova_sonic_duplicate_text (6451016) with main (8adb8f3)}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 6451016. Configure here.}

[mateo-berri]

LGTM