feat(provider): add Alibaba Cloud Token Plan provider

[lewisco]

TLDR

Problem this solves:

• Token Plan subscriptions have no native LiteLLM provider route
• Coding tools cannot route these models through the proxy

How it solves it:

• Adds alibaba_token_plan/ and 21 subscription models
• Supports chat, Messages, Responses, images, speech, realtime and video
• Reuses existing DashScope translation where the APIs match

Docs: BerriAI/litellm-docs#2085

User Flow

Before: Token Plan is missing from Add Model and requests fail

1. Open http://localhost:3000/models-and-endpoints and choose Add Model
2. Search Provider for Alibaba
3. See No providers found
4. With the shared wildcard config, POST http://localhost:4000/v1/chat/completions using chat.json returns HTTP 400 Invalid model name passed

After: the same page offers Token Plan and its models

1. Open http://localhost:3000/models-and-endpoints and choose Add Model
2. Search Provider for Alibaba
3. Select Alibaba Cloud Token Plan, then choose the wildcard or a model before entering a key
4. With the same wildcard config, POST http://localhost:4000/v1/chat/completions using chat.json returns HTTP 200 with pong

Relevant issues

Relates to #36150, where the QwenCloud docs note that Token Plan endpoints were not yet mapped

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Validation at e40d8af8f2, after merging main: make check and 629 focused tests passed. Earlier at 5eae427b48, 1,103 focused tests and the recursion detector passed. Nine regression cases fail with the dbc41e65d9 adapter and pass with the fix

Screenshots / Proof of Fix

Current tip is e40d8af8f2336fd9070ddb188c694fe539c57158. The 16 kHz realtime voice/tool proof ran at 6538becb9e on 2026-10-10; the later commits only change CI sharding, transcription format validation and video prompt precedence. Earlier voice/tool verification ran at 5eae427b48 on 2026-10-05. API/UI captures remain at 45d050e1c1, Terminal panes at 6ac150fc7b, and rechecks at 57a704962f and dbc41e65d9. Before remains at original merge base 02f61c9c420b9aa9de10ff673098ad7132b78f5b

The proof appendix holds commands, payloads, realtime client code, fixtures and supporting captures. Successful calls hit the real Token Plan API

model_list:
  - model_name: alibaba_token_plan/*
    litellm_params:
      model: alibaba_token_plan/*
      api_key: os.environ/ALIBABA_TOKEN_PLAN_API_KEY
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Start the proxy with LITELLM_LOCAL_MODEL_COST_MAP=true, the Token Plan key and a random master key in its environment. In a second shell:

B=http://localhost:4000
AUTH="Authorization: Bearer $LITELLM_MASTER_KEY"
post() {
  curl -sS --max-time 240 "$B/v1/$1" -H "$AUTH" \
    -H 'Content-Type: application/json' --data-binary "$2"
}

Interactive Claude Code uses ANTHROPIC_BASE_URL=http://localhost:4000, ANTHROPIC_AUTH_TOKEN=$LITELLM_MASTER_KEY, ANTHROPIC_MODEL=alibaba_token_plan/qwen3.8-max and ANTHROPIC_DEFAULT_HAIKU_MODEL=alibaba_token_plan/qwen3.8-flash. The appendix supplies the greeting fixture

Before (02f61c9)

Admin UI provider selection

1. Open http://localhost:3000/models-and-endpoints, choose Add Model, and search Provider for Alibaba
2. No providers found, shown in the Before image under User Flow

Claude Code

1. Start interactive Claude Code with the shared settings and ask it to read the greeting fixture, reproduce the whitespace failures, edit only greeting.py, then rerun python3 -m unittest -v

2. HTTP 400 anthropic_messages: Invalid model name passed in model=alibaba_token_plan/qwen3.8-max, before any tool call. The separate baseline proxy was stopped afterward

   

Chat completions

1. Run post chat/completions @chat.json, post chat/completions @chat-stream.json and post chat/completions @auto-tools.json
2. Each returns HTTP 400 Invalid model name passed, naming qwen3.8-max, qwen3.8-flash and auto respectively

Messages

1. Run post messages @messages.json and post messages @messages-stream.json
2. Both return HTTP 400 anthropic_messages: Invalid model name passed

Responses

1. Run post responses @responses.json
2. HTTP 400 /responses: Invalid model name passed in model=alibaba_token_plan/deepseek-v4-pro

Images

1. Run post images/generations @image-request.json and post images/generations @image-b64.json
2. Both return HTTP 400 Invalid model name passed for qwen-image-3.0-pro
3. Run curl -sS "$B/v1/images/edits" -H "$AUTH" -F model=alibaba_token_plan/wan2.7-image-pro -F 'image[]=@cube.png' -F 'image[]=@generated.png' -F 'prompt=Put both cubes side by side and make them blue' -F size=1024x1024
4. HTTP 400 Invalid model name passed for wan2.7-image-pro; this run used valid copied PNG fixtures

Speech and transcription

1. Run post audio/speech @speech.json, then curl -sS "$B/v1/audio/transcriptions" -H "$AUTH" -F model=alibaba_token_plan/qwen-audio-3.0-asr-flash -F file=@speech.mp3 -F language=en with a valid copied MP3 fixture
2. Both return HTTP 400 Invalid model name passed, naming the TTS and ASR models respectively

Realtime

1. Run the appendix's python realtime.py against ws://localhost:4000/v1/realtime?model=alibaba_token_plan/qwen-audio-3.0-realtime-plus
2. Voice and tool sessions return an error event: _arealtime: Invalid model name passed in model=alibaba_token_plan/qwen-audio-3.0-realtime-plus

Video

1. Run post videos @video-request.json, requesting one three-second 480P, 16:9 clip
2. HTTP 400 /videos: Invalid model name passed in model=alibaba_token_plan/happyhorse-1.1-t2v

Model list

1. Run curl -sS "$B/v1/models" -H "$AUTH" | jq '[.data[].id | select(startswith("alibaba_token_plan/"))] | length'
2. 0

After (6538bec)

Admin UI provider selection

1. At 45d050e1c1: open http://localhost:3000/models-and-endpoints, choose Add Model, and search Provider for Alibaba
2. Select Alibaba Cloud Token Plan, shown in the After image under User Flow
3. Open LiteLLM Model Name(s) before entering a provider key: the wildcard and 21 options are available, with the dropdown captured here

Claude Code

1. At 6ac150fc7b: start the same interactive task. Read, Edit and python3 -m unittest -v were pre-approved for this After run

2. Claude reads both files and runs the tests to reproduce the bug

   

3. It reproduces two failures, changes only greeting.py to use name.strip(), then reports Ran 3 tests and OK

   

4. Supporting Before transcript, After transcript and proxy logs retain their original revisions

Chat completions

1. At 45d050e1c1: run post chat/completions @chat.json
2. HTTP 200, {"model":"alibaba_token_plan/qwen3.8-max","content":"pong","total_tokens":91}
3. Run post chat/completions @chat-stream.json and post chat/completions @auto-tools.json
4. Streaming produces 1, 2, 3, 4, 5 and [DONE]; auto returns get_weather with {"city":"Paris"} and finish_reason: tool_calls
5. Earlier 57a704962f rerun: post chat/completions '{"model":"alibaba_token_plan/qwen3.8-max","messages":[{"role":"user","content":"Reply with exactly: pong"}]}'
6. HTTP 200, {"model":"alibaba_token_plan/qwen3.8-max","content":"pong","total_tokens":91}
7. Earlier dbc41e65d9 rerun: repeat step 5
8. HTTP 200, {"model":"alibaba_token_plan/qwen3.8-max","content":"pong","total_tokens":96}

Messages

1. At 45d050e1c1: run post messages @messages.json and post messages @messages-stream.json
2. Both return HTTP 200. Plain response: {"model":"alibaba_token_plan/glm-5.3","stop_reason":"end_turn","text":["pong"]}
3. Streaming includes message_start, 38 content_block_delta events and message_stop

Responses

1. At 45d050e1c1: run post responses @responses.json
2. HTTP 200, {"status":"completed","text":["pong"]}

Images

1. At 45d050e1c1: run post images/generations @image-request.json > image-result.json, then curl -sS -o generated.png "$(jq -r '.data[0].url' image-result.json)"
2. HTTP 200, one signed image URL; downloaded PNG is 1024 x 1024
3. Run post images/generations @image-b64.json
4. HTTP 400 Alibaba Token Plan image generation only returns image URLs; use response_format='url'
5. Run the same two-image edit command from Before, using the newly generated PNG
6. HTTP 200, one image URL; downloaded edit is a 1024 x 1024 PNG

Speech and transcription

1. At 45d050e1c1: run post audio/speech @speech.json > speech.mp3, then the same transcription command from Before with this MP3
2. Speech returns HTTP 200, audio/mpeg, 57,211 bytes
3. ASR returns HTTP 200, {"text":"Your build finished without errors at nine fifteen. ","usage":{"type":"duration","seconds":3.0},"duration":3.0}

Realtime

1. At 45d050e1c1: run the same python realtime.py, which sends 16 kHz PCM16, then performs a get_weather tool round-trip
2. Input transcript: What is the capital of France? Answer in one short sentence. Reply: The capital of France is Paris. Audio: 103,680 bytes, 2.16 seconds at 24 kHz; status completed
3. At 6538becb9e, after merging fix(realtime): only inject response.create when a transcript guardrail suppressed VAD auto-response #43791, the same voice turn with a regenerated recording of the same question replies The capital of France is Paris., 157,440 bytes, 3.28 seconds at 24 kHz, completed, with error events: []. No Cannot create response while another response is in progress event appears, and the weather tool turn also completes
4. Tool call: get_weather {"city":"Paris"}. Final answer: The weather in Paris is 18°C with light rain. Status completed
5. Earlier 57a704962f rerun: connect to ws://localhost:4000/v1/realtime?model=alibaba_token_plan/qwen-audio-3.0-realtime-plus with the shared Authorization header and send session.update with audio.input.format={"type":"audio/pcm","rate":16000}, then an instructions-only update and the same voice/tool turns
6. Voice: The capital of France is Paris., 111,360 bytes, 2.32 seconds at 24 kHz, completed, no errors. get_weather({"city":"Paris"}) and its final reply complete
7. On separate new connections, send {"type":"session.update","session":{"instructions":"Answer briefly."}}, then either {"type":"input_audio_buffer.append","audio":"AAAAAA=="} or {"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_audio","audio":"AAAAAA=="}]}}
8. Both undeclared-audio cases close with client code 1006
9. On another connection, send {"type":"session.update","session":{"type":"realtime","audio":{"input":{"format":{"type":"audio/pcm","rate":24000}}}}}
10. The connection closes without a WebSocket close frame
11. Earlier dbc41e65d9 rerun: repeat steps 5-10
12. Voice: 111,360 bytes at 24 kHz, completed, no errors. Weather tool/reply: completed. All three rejected sessions: client code 1006, no close frame
13. At 5eae427b48, repeat the declared 16 kHz voice/tool requests from step 5
14. Voice: The capital of France is Paris., 111,360 bytes at 24 kHz, completed. Tool reply: The weather in Paris is 18°C with light rain., completed. No error events

Video

1. At 45d050e1c1: run post videos @video-request.json > video-result.json; save VIDEO_ID=$(jq -r '.id' video-result.json)
2. HTTP 200, initially queued; polling curl -sS "$B/v1/videos/$VIDEO_ID" -H "$AUTH" ends with completed, seconds: "3", video_resolution: "480p"
3. Run curl -sS "$B/v1/videos/$VIDEO_ID/content" -H "$AUTH" -o t2v.mp4
4. HTTP 200, 719,080 bytes; ffprobe reports H.264 848x480, 3.041667 seconds, plus AAC audio. Only one video was generated

Model list

1. At 45d050e1c1: run the same GET and jq command from Before
2. 21, both at 45d050e1c1 and in the earlier 6ac150fc7b smoke
3. The earlier 57a704962f rerun returned HTTP 200, 21
4. Historical dbc41e65d9 rerun of the same GET/jq: HTTP 200, 21

Type

New Feature

Caveats

Medium

• Transcription guardrails are not supported on Token Plan realtime; sessions requesting them reject audio

• Audio is rejected until the session declares 16 kHz input; declaring 24 kHz closes the session

  • 24 kHz input adaptation is tracked in [Feature]: Accept 24 kHz input audio on Alibaba Token Plan realtime #44602

• Invalid chat keys return HTTP 401 typed as BadRequestError

  • Fixed separately in fix(exceptions): classify OpenAI-compatible 401 responses as authentication errors #44582

Low

• Documentation CI depends on docs(providers): add Alibaba Cloud Token Plan litellm-docs#2085 merging

• Earlier rust-wheel failed upstream at 45 MB, now raised to 48 MB

  • Earlier main failure

• Spend shows $0 because the subscription bills credits

• Images return URLs; b64_json fails unless drop_params is enabled

• Transcription returns json or text; srt, vtt and verbose_json fail unless drop_params is enabled

• Alibaba's terms limit Token Plan to interactive coding tools

• Image-to-video and reference-to-video were not rerun in this proof

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Large new provider surface (realtime WebSocket translation, signed media downloads, provider routing) touches core HTTP/TTS paths; mistakes could mis-route credentials or break unrelated TTS async handling.

Overview
Adds a first-class alibaba_token_plan provider for Alibaba Cloud Token Plan subscriptions, wired through LiteLLM’s provider registry, proxy credential fields, dashboard “Add Model” UI, model catalog, and CI unit-test shard.

Routing uses dedicated ALIBABA_TOKEN_PLAN_API_KEY / API_BASE resolution (with a special case so JSON openai-like config does not swallow the provider). Chat builds on DashScope translation; multimodal paths add native transforms for Anthropic /messages, transcription, TTS (URL fetch via new async_transform_text_to_speech_response), image gen/edit, realtime (OpenAI event shaping, 16 kHz PCM guardrails), and async video create/poll/download. 21 subscription models and endpoint capability metadata are registered; README lists the provider.

^{Reviewed by Cursor Bugbot for commit e40d8af. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues remain from this review.

Summary

Adds Alibaba Cloud Token Plan routing, 21 models, dashboard selection, and chat, Messages, Responses, media, and realtime support.

Since the previous review:

• LiteLLM routes Token Plan model names to Alibaba Cloud.
• Token Plan models work with chat, Messages, and Responses requests.
• Token Plan models can generate and edit images.
• Token Plan models transcribe audio and return generated speech.
• Token Plan models support realtime audio, text, and tools.
• Token Plan models can create and retrieve generated videos.

No new actionable issues were found. The earlier credential-redirection concern was withdrawn after lewisco explained the proxy's default protection. The earlier realtime audio-rate finding is addressed by requiring declared 16 kHz input.

_{Reviews (5) · Last reviewed commit: "fix(provider): keep the guardrail-checke..." · Reviewed by Greptile}

[codspeed]

Merging this PR will not alter performance

⚠️ 1 benchmark spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• test_completion_streaming

✅ 31 untouched benchmarks

---

_{Comparing lewisco:litellm_alibaba_token_plan (e40d8af) with main (7ff5010)}

[codecov]

Codecov Report

❌ Patch coverage is 93.35727% with 37 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...llms/alibaba_token_plan/realtime/transformation.py	90.80%	16 Missing ⚠️
...m/llms/alibaba_token_plan/videos/transformation.py	88.13%	14 Missing ⚠️
...a_token_plan/audio_transcription/transformation.py	95.00%	3 Missing ⚠️
...libaba_token_plan/text_to_speech/transformation.py	95.16%	3 Missing ⚠️
...llm/llms/alibaba_token_plan/chat/transformation.py	85.71%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[lewisco]

@greptileai Please re-review the updated provider branch at 57a7049, including the realtime audio validation fix and clarified credential guard

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[lewisco]

bugbot run

[lewisco]

@greptileai

[cursor]

Stale Bugbot comment from a previous run.

[lewisco]

@mateo-berri ready for review: Greptile 5/5, Veria clean, required checks green; documentation checks wait on docs PR BerriAI/litellm-docs#2085

[lewisco]

@greptileai

[lewisco]

@greptileai

[lewisco]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit e40d8af. Configure here.}