feat(realtime): add Meta Muse Voice transcription

[seyeong-han]

TLDR

Problem this solves:

• LiteLLM cannot stream Meta Muse Voice transcription
• Generic realtime providers cannot forward raw binary PCM

How it solves it:

• Adds meta/muse-voice-transcribe-1.0 realtime routing
• Maps OpenAI transcription events to Muse binary streaming
• Preserves duration usage without logging private content

User Flow

Before: a developer cannot use Muse Voice through LiteLLM's realtime endpoint

1. They connect to WSS /v1/realtime?model=meta/muse-voice-transcribe-1.0&intent=transcription
2. LiteLLM has no Muse realtime route, so transcription cannot start
3. Their application must integrate Meta's provider protocol directly

After: the same developer receives OpenAI-compatible transcription events from Muse

1. The proxy admin sets MODEL_API_KEY and starts the proxy
2. The developer connects to WSS /v1/realtime?model=meta/muse-voice-transcribe-1.0&intent=transcription
3. They send session.update, paced PCM16 audio, and input_audio_buffer.commit
4. They receive transcription deltas, a completed transcript, and duration usage

Relevant issues

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The focused test files covering my change pass locally
• My PR passes local lint, type, and repository budget checks
• My PR's scope is as isolated as possible
• I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Shared setup:

• Model: meta/muse-voice-transcribe-1.0
• Route: WSS /v1/realtime?model=meta/muse-voice-transcribe-1.0&intent=transcription
• Audio: repository weather_question_24k.wav, mono PCM16 at 24 kHz
• Upstream: live wss://api.meta.ai/v1/asr/realtime

Before (d2fe8af)

1. Started LiteLLM from the staging base with the Muse model route
2. The base contained no Meta Muse realtime handler or model registration
3. A proxy-to-Muse realtime transcription could not be performed

After (298a729)

1. Started LiteLLM with LITELLM_LOCAL_MODEL_COST_MAP=True and server-side MODEL_API_KEY
2. Connected through the public realtime route and streamed the 24 kHz fixture
3. Received session.created, session.updated, speech_started, transcription deltas, input_audio_buffer.committed, speech_stopped, and completed
4. Completed transcript: What is the weather in Paris??
5. Provider-confirmed duration usage: 1.36 seconds
6. Item IDs remained consistent and the WebSocket closed with code 1000

Local validation:

• 185 passed, 9 warnings in the focused realtime suite
• Ruff formatting and lint passed
• Strict Ruff, type-discipline, test-quality, and BasedPyright budget gates passed
• JSON parsing, model registry synchronization, and git diff --check passed

Type

🆕 New Feature

Caveats (if any)

Medium

• Supports realtime transcription only; batch transcription is excluded
• Supports mono PCM16 at 16 kHz or 24 kHz
• No public Muse price is recorded; usage duration is still logged

Low

• Conversational realtime response events are rejected
• MODEL_API_KEY is preferred; META_API_KEY remains a fallback

Final Attestation

• The tests cover protocol edge cases and real-world transcription behavior

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing seyeong-han:litellm_meta_muse_voice_realtime (298a729) with litellm_internal_staging (7114302)}

[greptile-apps]

Greptile Summary

The PR adds Meta Muse Voice realtime transcription routing and translates between OpenAI-compatible transcription events and Muse’s binary PCM protocol.

• Adds Muse session validation, provider handshake, paced audio forwarding, event transformation, and duration usage reporting.
• Adds privacy-preserving realtime logging and model/provider registration.
• Adds focused coverage for routing, protocol transformation, streaming, URL validation, and lifecycle behavior.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/meta/realtime/handler.py	Implements the Muse WebSocket adapter, secure endpoint validation, handshake, paced PCM forwarding, bounded buffering, and connection cleanup; no eligible unresolved follow-up issue remains.
litellm/llms/meta/realtime/transformation.py	Validates Muse session configuration and translates provider transcription events into OpenAI-compatible events.
litellm/litellm_core_utils/realtime_streaming.py	Adds provider-supplied usage flushing and content-sanitized logging for privacy-sensitive transcription sessions.
litellm/realtime_api/main.py	Integrates Meta Muse into realtime provider dispatch and credential resolution.
tests/test_litellm/llms/meta/realtime/test_meta_realtime_handler.py	Exercises adapter validation and lifecycle behavior with injected transports; the insecure URL literal is explicitly verified as rejected and cannot make a network request.
tests/test_litellm/llms/meta/realtime/test_meta_realtime_transformation.py	Covers session parsing and Muse-to-OpenAI transcription event transformation.

_{Reviews (2): Last reviewed commit: "fix(realtime): bound Muse audio before d..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 84.44194% with 138 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/meta/realtime/handler.py	81.17%	80 Missing ⚠️
litellm/llms/meta/realtime/transformation.py	86.43%	54 Missing ⚠️
litellm/litellm_core_utils/realtime_streaming.py	90.00%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[seyeong-han]

Companion documentation PR: BerriAI/litellm-docs#1138

It documents MODEL_API_KEY, the Muse Voice model, /v1/realtime, PCM requirements, and the realtime-only scope. The exact tests/documentation_tests/test_env_keys.py gate passes when that docs branch is checked out. The current code-quality failure reads the docs repository default branch, so it will remain expected until the companion docs PR lands.

[seyeong-han]

@greptileai Please re-review the latest commit. The oversized base64 allocation finding is fixed in 298a729; the direct connector rationale is documented in the inline reply.