feat(sail): add Sail as a provider with service_tier mapped to its completion window

[devin-ai-integration]

TLDR

Problem this solves:

• sail/<model> is not a known provider, so every Sail request fails
• Sail picks speed and price with metadata.completion_window, which LiteLLM has no way to set or bill

How it solves it:

• Registers Sail as an OpenAI-compatible provider for chat, Responses and /v1/messages
• Adds its 12 models to the cost map with asap, balanced and flex prices
• Maps service_tier to Sail's window: flex to flex, balanced to balanced, default and priority to asap
• Bills the price columns of the window that was sent, on every request path
• Rejects a tier or window LiteLLM could not bill with a 400 (dropped under drop_params)
• Non-Sail behavior checked unchanged: 1138 request bodies and 102,720 costs, base vs branch

Intentional product change: balanced is a new service_tier value, and model info output such as /model/info gains three price fields (input_cost_per_token_balanced, output_cost_per_token_balanced, cache_read_input_token_cost_balanced) that are null for every non-Sail model

User Flow

Before: a proxy admin adds a Sail model and every request to it fails

1. They add model: sail/zai-org/GLM-5.3 with api_key: os.environ/SAIL_API_KEY to the proxy config and restart
2. They send POST http://localhost:4000/v1/chat/completions with "model": "sail-glm" and "service_tier": "flex"
3. The proxy answers 400 because it does not know the sail provider

After: the same request runs on Sail's flex window and is billed at Sail's flex price

1. They add model: sail/zai-org/GLM-5.3 with api_key: os.environ/SAIL_API_KEY to the proxy config and restart
2. They send POST http://localhost:4000/v1/chat/completions with "model": "sail-glm" and "service_tier": "flex"
3. They get a 200, and x-litellm-response-cost equals the flex price times the returned token counts
4. "balanced" bills the balanced price and no tier bills the asap price, the same way on POST /v1/responses
5. "service_tier": "bogus" gets a 400 from LiteLLM that names the supported tiers, and nothing reaches Sail

Design decisions

• service_tier is the only knob, because it is the field LiteLLM's billing reads
  • default and priority send asap, since Sail has nothing faster
  • auto sends no window, so Sail uses its own default, which is asap
  • Tier names are case-insensitive: FLEX works like flex
• Anything else in service_tier gets a 400 from LiteLLM before Sail is called
  • This covers unknown strings like "bogus" and non-strings like 5
  • Under drop_params the value is dropped and the call runs and bills at asap
• On /v1/responses a caller may also set metadata.completion_window directly, because metadata there is a real OpenAI field that reaches Sail
  • standard, Sail's alias for balanced, is accepted and billed as balanced
  • A window that disagrees with service_tier is a 400
• On /v1/chat/completions a window in metadata or extra_body.metadata is a 400
  • Chat billing only sees service_tier, so that window would run at one price and bill at another
• service_tier inside extra_body is a 400 on both endpoints, for the same reason
• A caller's other extra_body.metadata keys are kept and merged with the window instead of replacing it
• /v1/messages passes through like every other provider on that path, so service_tier is ignored there
• Sail rejects stop, seed, frequency_penalty, presence_penalty, logit_bias, logprobs and top_logprobs, so LiteLLM rejects them first and drop_params can strip them
• max_tokens and max_completion_tokens pass through as sent, since Sail accepts and enforces both (checked live below)
• balanced joins the shared ServiceTier enum with _balanced price columns; only Sail rows carry them, so every other model prices exactly as before
• Sail is listed in the Add Model form and the dashboard model picker

Linear ticket

Resolves LIT-8503

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Every call below hits the real Sail API through a local proxy, run on 2026-09-26. The proxy config has two deployments on sail/zai-org/GLM-5.3: sail-glm, and sail-glm-drop with drop_params: true. Both use api_key: os.environ/SAIL_API_KEY. The proxy was started with LITELLM_MODE=PRODUCTION LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config sail.yaml --port 4001, and $LITELLM_MASTER_KEY is a random local key

Costs are checked by hand against the sail/zai-org/GLM-5.3 row: asap (base) 9.8e-07 input, 1.8e-07 cached, 3.08e-06 output; balanced 5e-07, 1.2e-07, 2.5e-06; flex 4e-07, 8e-08, 1.8e-06

Before (e065a25)

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":64,"service_tier":"flex"}'
2. HTTP/1.1 400 Bad Request, litellm.BadRequestError: You passed in model=sail-glm. There are no healthy deployments for this model, because main does not know the sail provider

After (c38a38e)

chat, no service_tier

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":256}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00080612, usage 18 prompt (0 cached), 256 completion
3. 18 x 9.8e-07 + 256 x 3.08e-06 = 0.00080612, the asap price

chat, service_tier flex

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":256,"service_tier":"flex"}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.0004122, usage 18 prompt (0 cached), 225 completion
3. 18 x 4e-07 + 225 x 1.8e-06 = 0.0004122, the flex price

chat, service_tier balanced

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":256,"service_tier":"balanced"}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00050754, usage 18 prompt (17 cached), 202 completion
3. 1 x 5e-07 + 17 x 1.2e-07 + 202 x 2.5e-06 = 0.00050754, the balanced price

chat, unknown service_tier

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":64,"service_tier":"bogus"}'
2. HTTP/1.1 400 Bad Request, x-litellm-response-cost: 0, litellm.UnsupportedParamsError: sail does not support service_tier='bogus'. Supported values: auto, default, priority, flex, balanced.

chat, non-string service_tier

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":64,"service_tier":5}'
2. HTTP/1.1 400 Bad Request, litellm.UnsupportedParamsError: sail does not support service_tier=5. Supported values: auto, default, priority, flex, balanced.

chat, unknown service_tier under drop_params

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm-drop","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":256,"service_tier":"bogus"}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00079252, usage 18 prompt (17 cached), 256 completion
3. 1 x 9.8e-07 + 17 x 1.8e-07 + 256 x 3.08e-06 = 0.00079252, dropped and billed at asap

chat, a param Sail rejects

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Say hi in five words."}],"max_tokens":64,"stop":["x"]}'
2. HTTP/1.1 400 Bad Request, litellm.UnsupportedParamsError: sail does not support parameters: ['stop'], for model=zai-org/GLM-5.3

chat, max_tokens passes through

1. curl -si localhost:4001/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","messages":[{"role":"user","content":"Write a 200 word story about a cat."}],"max_tokens":16}'
2. HTTP/1.1 200 OK, usage 16 completion tokens, finish_reason: length, so Sail enforced the limit

responses, metadata.completion_window flex

1. curl -si localhost:4001/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","input":"Say hi in five words.","max_output_tokens":256,"metadata":{"completion_window":"flex"}}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00028256, usage 18 input (17 cached), 156 output, and Sail echoes metadata.completion_window: flex
3. 1 x 4e-07 + 17 x 8e-08 + 156 x 1.8e-06 = 0.00028256, the flex price

responses, service_tier balanced

1. curl -si localhost:4001/v1/responses -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"model":"sail-glm","input":"Say hi in five words.","max_output_tokens":256,"service_tier":"balanced"}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00064254, usage 18 input (17 cached), 256 output, and Sail echoes metadata.completion_window: standard, its internal name for balanced
3. 1 x 5e-07 + 17 x 1.2e-07 + 256 x 2.5e-06 = 0.00064254, the balanced price

messages, plain

1. curl -si localhost:4001/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -d '{"model":"sail-glm","max_tokens":256,"messages":[{"role":"user","content":"Say hi in five words."}]}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00080612, usage 18 input, 256 output
3. 18 x 9.8e-07 + 256 x 3.08e-06 = 0.00080612, the asap price

messages, service_tier flex is ignored

1. curl -si localhost:4001/v1/messages -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -H "anthropic-version: 2023-06-01" -d '{"model":"sail-glm","max_tokens":256,"service_tier":"flex","messages":[{"role":"user","content":"Say hi in five words."}]}'
2. HTTP/1.1 200 OK, x-litellm-response-cost: 0.00068292, usage 18 input, 216 output
3. 18 x 9.8e-07 + 216 x 3.08e-06 = 0.00068292, asap, as for every provider on this path

Type

🆕 New Feature

Caveats (if any)

Medium

• service_tier on /v1/messages is ignored, like for every provider on that path
  • The call runs and bills at asap; use chat or Responses for flex and balanced
• litellm.completion_cost(completion_response=resp) called by hand on a flex chat response prices it at asap
  • The logged cost, spend row and x-litellm-response-cost header are right; follow-up is LIT-8651

Low

• sail/openai/gpt-oss-120b has one price and sail/Qwen/Qwen3.6-35B-A3B has no balanced price on Sail's pricing page (checked 2026-09-25), so missing tiers bill the base price
• New transform_extra_body hook on the chat and Responses base configs
  • Its default returns extra_body unchanged, and only Sail overrides it
• Sail has no transcription, speech or image endpoints, so those calls get Sail's own 404

QA runbook

Prerequisites: a proxy with SAIL_API_KEY set, STORE_MODEL_IN_DB=True and Postgres for the spend rows. The tests pick the cheapest sail/ row whose asap, balanced and flex prices all differ from /model/info, and register their own deployment through /model/new

• tests/e2e/llm_translation/test_sail_e2e.py::TestSailChatCompletions::test_service_tier_bills_the_matching_completion_window - flex, balanced and auto each bill that window's price columns
  • POST /model/new with {"model_name": "sail-e2e", "litellm_params": {"model": "sail/zai-org/GLM-5.3", "api_key": "os.environ/SAIL_API_KEY"}}
  • POST /v1/chat/completions with "model": "sail-e2e" three times: "service_tier": "flex", "balanced", "auto"
  • Expect 200 each time, x-litellm-response-cost equal to (prompt - cached) x input + cached x cache read + completion x output at that tier's columns, and the spend row equal to the header
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
• tests/e2e/llm_translation/test_sail_e2e.py::TestSailChatCompletions::test_unknown_service_tier_is_rejected - a tier Sail has no window for is a 400 from LiteLLM
  • POST /v1/chat/completions with "service_tier": "bogus"
  • Expect 400 with a message naming service_tier
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
• tests/e2e/llm_translation/test_sail_e2e.py::TestSailResponses::test_flex_completion_window_bills_flex_rates - a caller's flex window on Responses bills flex
  • POST /v1/responses with "model": "sail-e2e", "input": "hi", "metadata": {"completion_window": "flex"}
  • Expect 200, the cost header at flex columns times usage, and the spend row equal to the header
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky
• tests/e2e/llm_translation/test_sail_e2e.py::TestSailMessages::test_plain_call_returns_a_message - /v1/messages works on Sail
  • POST /v1/messages with an Anthropic body, "model": "sail-e2e", "max_tokens": 16
  • Expect 200 with a text block and output token usage
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_add_sail_provider (6c7baaa) with main (4d7aa89)}

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds Sail as a new LLM provider with service tier pricing.

The PR appears safe to merge based on the reviewed changes and resolved prior findings.

Summary

The PR registers Sail for chat, Responses, and Messages, translates supported service tiers into completion windows, and adds tier-specific pricing, model metadata, UI registration, and tests. The changes since the previous review preserve both token-limit fields as supplied and make the live billing tests use deployment-specific rates.

_{Reviews (19) · Last reviewed commit: "feat(sail): add Sail as a provider with ..."}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[shrey-berri]

@greptileai

---

Generated by Claude Code

[shrey-berri]

bugbot run

---

Generated by Claude Code

[cursor]

Stale Bugbot comment from a previous run.

[shrey-berri]

@greptileai

---

Generated by Claude Code

[shrey-berri]

bugbot run

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit db710f9. Configure here.}

[mateo-berri]

LGTM. Thanks!

[mateo-berri]

@greptileai

[shrey-berri]

@greptileai please re-review the full PR diff against main. The last pass compared across a rebase and counted main's commits as this PR

---

Generated by Claude Code