Skip to content

Update Vertex AI Text to Speech doc to show use of audio#20255

Merged
Sameerlite merged 1 commit intomainfrom
litellm_tts_doc
Feb 2, 2026
Merged

Update Vertex AI Text to Speech doc to show use of audio#20255
Sameerlite merged 1 commit intomainfrom
litellm_tts_doc

Conversation

@Sameerlite
Copy link
Collaborator

Relevant issues

Fixes #18178

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

@vercel
Copy link

vercel bot commented Feb 2, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Building Building Preview, Comment Feb 2, 2026 10:46am

Request Review

@Sameerlite Sameerlite merged commit 15cec5a into main Feb 2, 2026
8 of 52 checks passed
@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 2, 2026

Greptile Overview

Greptile Summary

This PR updates the Vertex AI Text to Speech documentation to clarify that when using Gemini TTS models via LiteLLM Proxy, users must include allowed_openai_params to enable the audio and modalities parameters.

Changes

  • Added note in the warning section explaining the requirement for allowed_openai_params when using LiteLLM Proxy
  • Updated the curl example to include "allowed_openai_params": ["audio", "modalities"] in the request body
  • Updated the OpenAI Python SDK example to include extra_body={"allowed_openai_params": ["audio", "modalities"]}

Context

This addresses issue #18178. The allowed_openai_params parameter is necessary because:

  1. LiteLLM validates which OpenAI parameters are supported by each provider/model
  2. By default, if a parameter is not explicitly marked as supported, LiteLLM will raise an UnsupportedParamsError
  3. The allowed_openai_params field allows users to dynamically override this validation and explicitly permit certain parameters
  4. For Gemini TTS models accessed through the proxy, the audio and modalities parameters need to be explicitly allowed to pass through

The documentation change is consistent with LiteLLM's existing pattern for handling provider-specific parameters through the proxy, as documented in /docs/completion/drop_params.md.

Confidence Score: 4/5

  • This PR is safe to merge with low risk - it's a documentation-only change that clarifies usage
  • Documentation change is accurate and helpful. The score is 4/5 rather than 5/5 because: (1) the PR lacks tests as noted in the pre-submission checklist, though tests may not be strictly necessary for documentation changes, and (2) it would be helpful to verify that the documented pattern actually works as described
  • No files require special attention

Important Files Changed

Filename Overview
docs/my-website/docs/providers/vertex_speech.md Added allowed_openai_params requirement for audio parameters when using Gemini TTS via proxy

Sequence Diagram

sequenceDiagram
    participant User
    participant OpenAI_SDK as OpenAI SDK Client
    participant LiteLLM_Proxy as LiteLLM Proxy
    participant Vertex_AI as Vertex AI Gemini TTS
    
    User->>OpenAI_SDK: Create chat completion request
    Note over User,OpenAI_SDK: messages, modalities=["audio"]<br/>audio={"voice": "Kore", "format": "pcm16"}
    OpenAI_SDK->>LiteLLM_Proxy: POST /v1/chat/completions
    Note over OpenAI_SDK,LiteLLM_Proxy: extra_body={"allowed_openai_params": ["audio", "modalities"]}
    
    LiteLLM_Proxy->>LiteLLM_Proxy: Check allowed_openai_params
    Note over LiteLLM_Proxy: Validates "audio" and "modalities"<br/>are in allowed list
    
    LiteLLM_Proxy->>LiteLLM_Proxy: Transform audio params to speechConfig
    Note over LiteLLM_Proxy: Maps OpenAI audio format<br/>to Vertex AI speechConfig
    
    LiteLLM_Proxy->>Vertex_AI: Generate content with audio
    Note over LiteLLM_Proxy,Vertex_AI: speechConfig={voiceConfig={prebuiltVoiceConfig={voiceName: "Kore"}}}<br/>responseModalities=["AUDIO"]
    
    Vertex_AI-->>LiteLLM_Proxy: Audio response (pcm16)
    LiteLLM_Proxy-->>OpenAI_SDK: OpenAI-format response
    OpenAI_SDK-->>User: Audio content
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: tts issue with gemini tts models in vertex ai

1 participant