feat(decisions): add hosted_vllm provider

[devin-ai-integration]

TLDR

Problem this solves:

• vLLM now serves the System One body at /v1/systemone
• LiteLLM rejects hosted_vllm on /v1/decisions and /v1/systemone
• /health probes Decisions deployments with a noul question, which vLLM rejects

How it solves it:

• New HostedVLLMDecisionsConfig, registered in get_provider_decisions_config
• Reads HOSTED_VLLM_API_BASE (required) and HOSTED_VLLM_API_KEY (optional)
• New BaseDecisionsConfig.health_check_questions, which vLLM overrides with a choice probe

User Flow

Before: a proxy admin who points a mode: evaluation deployment at a vLLM server gets a 400 on every Decisions call, and the deployment shows as unhealthy

1. They add model: hosted_vllm/Qwen/Qwen3-0.6B, api_base: http://vllm:8000/v1, model_info: {mode: evaluation}
2. A client sends POST https://litellm-domain/v1/decisions with a choices question
3. It gets 400 Unknown Decisions provider 'hosted_vllm', and POST /v1/systemone returns the same error
4. GET https://litellm-domain/health?model=qwen-vllm returns 503 with the deployment unhealthy

After: the same calls reach vLLM and come back as typed answers, and the deployment shows as healthy

1. They add the same deployment
2. A client sends POST https://litellm-domain/v1/decisions with a choices question
3. It gets 200 with a typed choice answer, and POST /v1/systemone with a Jev choice question also returns 200
4. GET https://litellm-domain/health?model=qwen-vllm returns 200 with healthy_count: 1

Linear ticket

Resolves LIT-9367

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

A real vLLM server, a CPU image built from vllm main 702ce313f (vllm-project/vllm#59299 is not in v0.31.0), serves Qwen/Qwen3-0.6B on :8000. The proxy config is the same on both sides:

model_list:
  - model_name: qwen-vllm
    litellm_params:
      model: hosted_vllm/Qwen/Qwen3-0.6B
      api_base: http://localhost:8000/v1
    model_info:
      mode: evaluation

Before (0ae62c5)

/v1/decisions

1. curl -X POST http://localhost:4002/v1/decisions with an OpenAI-spec choices question
2. HTTP 400 {"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'hosted_vllm'. Supported providers: openai, openrouter, perplexity, typesafe, strands_decider, cloudflare ..."}}

/health

1. curl "http://localhost:4002/health?model=qwen-vllm"
2. HTTP 503 {"healthy_endpoints":[],"unhealthy_endpoints":[{"model":"hosted_vllm/Qwen/Qwen3-0.6B","error":"litellm.BadRequestError: Unknown Decisions provider 'hosted_vllm' ..."}],"healthy_count":0,"unhealthy_count":1}

After (7b7fa51)

/v1/decisions

1. curl -X POST http://localhost:4003/v1/decisions with the same choices question
2. HTTP 200 {"model":"Qwen/Qwen3-0.6B","answers":[{"type":"choice","name":"is_defect","choice":"yes","probabilities":[{"value":"yes","probability":0.8175744828610095},{"value":"no","probability":0.18242551713899047}],"confidence":0.8159867393125323}],"usage":{"input_tokens":45,"output_tokens":1,"total_tokens":46,...}}
3. curl -X POST http://localhost:4003/v1/systemone with a Jev choice question
4. HTTP 200 {"model":"Qwen/Qwen3-0.6B","answers":{"is_urgent":{"type":"choice","choice":"yes","confidence":0.9649145337527952,"probabilities":{"yes":0.9669140238756562,"no":0.03308597612434374}}},"usage":{"input_tokens":61,"output_tokens":1},"id":"decision-890b2c7fa31bb494","object":"structured_decision","created":1791520584,"diagnostics":{...}}
5. The SpendLogs row for that call shows custom_llm_provider=hosted_vllm, call_type=adecisions, api_base=http://localhost:8000/v1/systemone, status=success

/health

1. curl "http://localhost:4003/health?model=qwen-vllm"
2. HTTP 200 {"healthy_endpoints":[{"api_base":"http://localhost:8000/v1","model":"hosted_vllm/Qwen/Qwen3-0.6B",...}],"unhealthy_endpoints":[],"healthy_count":1,"unhealthy_count":0}

Type

🆕 New Feature

Caveats (if any)

Medium

• vLLM answers only choice questions today
  • predicate and levels return a 400: unknown question type 'noul'; supported: ['choice']
• Needs a vLLM build newer than v0.31.0 and a Qwen3 or Qwen3.5 model; other models get a 501

Low

• HostedVLLMDecisionsConfig.resolve_api_key drops the fake-api-key placeholder that hosted_vllm chat injects, so no Authorization header is sent
• Docs row is in a separate litellm-docs PR

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/3d0a4b725c1d4d65968372a35704cdd2
Open in Devin Desktop: https://app.devin.ai/desktop/session/3d0a4b725c1d4d65968372a35704cdd2?variant=devin
Requested by: @kerry-berri

---

Note

Devin errored when opening this Pull Request as kerry-berri.
As a fallback, Devin opened this PR as itself.

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

kerry seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with the previous finding fully addressed and no new blocking issues

Summary

Adds hosted_vllm support for Decisions calls and uses a choice question for its health checks

• Hosted vLLM models can answer Decisions requests through /v1/systemone.
• Health checks send the question format each Decisions provider expects.

_{Reviews (2) · Last reviewed commit: "refactor(decisions): move the hosted_vll..." · Reviewed by Greptile}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_hosted_vllm_decisions (7b7fa51) with main (4b975e6)¹}

Footnotes

1. No successful run was found on main (72ab736) during the generation of this report, so 4b975e6 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai