feat(decisions): backport the decisions and Jev follow-ups to rc/1.105.0 without the playground

[devin-ai-integration]

TLDR

Backports the decisions and Jev work merged on main since #45189 onto rc/1.105.0, without the decisions playground, so the 1.105.0 rc serves every decisions provider and SDK shape the launch ships with

Problem this solves:

• rc/1.105.0 only has the openai, perplexity, typesafe, openrouter, cloudflare and strands_decider decisions providers
• It has no hosted_vllm or azure_ai Microsoft-Decision-1 provider, no AgentCore runtime ARN support, no litellm.systemone
• OpenRouter decisions calls are billed from the cost map instead of the provider-reported usage.cost
• TypeSafe and Strands Decider are missing from the Add Model provider list, and Swagger UI does not describe the two decisions routes

How it solves it:

• Cherry-picks the 16 main commits listed under Picks with git cherry-pick -x, adapted where rc/1.105.0 lacks what they build on
• Regenerates schema.d.ts on the line instead of merging the generated file by hand
• Leaves the decisions playground (feat(ui): let admins find, call, and test decision models from the Admin UI #45654, feat(ui): link the decision models banner to the supported providers docs and rename the System One playground to Decisions #45732, fix(ui): drop the decision models sublabel from the Add Model provider dropdown #45755) out, as asked

User Flow

Before: a proxy admin on 1.105.0 cannot route a decisions request to a hosted vLLM or Azure decider

1. They add hosted_vllm/my-decider or azure_ai/Microsoft-Decision-1 to the config and start the proxy
2. They send POST http://localhost:4000/v1/decisions with that model, an input and yes/no questions
3. They get HTTP 400 "Unknown Decisions provider", and the Add Model page has no TypeSafe or Strands Decider provider

After: the same request is answered, and the new providers show up everywhere the old ones did

1. They add the same models to the config and start the proxy
2. The same POST http://localhost:4000/v1/decisions returns HTTP 200 with one answer per question and a usage block, billed at the line's price for that model
3. http://localhost:4000/ui/?page=models lists TypeSafe and Strands Decider under Add Model, and http://localhost:4000/#/v1/decisions in Swagger UI documents both routes
4. litellm.systemone(...) and litellm.decisions(...) work from the SDK with the same shapes as main

Picks

• 79f62db feat(decisions): add the OpenAI Decisions spec types and the System One translation (feat(decisions): add the OpenAI Decisions spec types and the System One translation #45129)
• 0e09e99 refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler (refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler #45130)
• a7b04a8 test(decisions): post System One bodies to /v1/systemone in the translation bases (test(decisions): post System One bodies to /v1/systemone in the translation bases #45327)
• 6e54dce test: realign stale decisions and lens tests with merged behaviour (test: realign stale decisions and lens tests with merged behaviour #45346)
• 3a3b8ca feat(ui): add TypeSafe and Strands Decider to the Add Model provider list (feat(ui): add TypeSafe and Strands Decider to the Add Model provider list #45335)
• 0e48048 chore(decisions): remove the System One converters and OpenAI spec types left dead by feat(decisions): add OpenAI as a Decisions provider behind a shared decisions format #45214 (chore(decisions): remove the System One converters and OpenAI spec types left dead by #45214 #45442)
• 61e5f2d feat(decisions): add hosted_vllm provider (feat(decisions): add hosted_vllm provider #45501)
• de6a071 feat(azure): add azure_ai/Microsoft-Decision-1 (feat(azure): add azure_ai/Microsoft-Decision-1 #45639)
• 465f20f fix(openrouter): bill responses, decisions and pass-through from usage.cost and always allow reasoning params (fix(openrouter): bill Responses, Decisions and pass-through from OpenRouter usage.cost, always allow reasoning params #45640), decisions and pass-through billing only
• 2fcc780 feat(azure_ai): support Microsoft-Decision-1 on the decisions API (feat(azure_ai): support Microsoft-Decision-1 on the decisions API #45673)
• 546402c fix(decisions)!: refuse safety_identifier on providers that cannot take it unless drop_params drops it (fix(decisions)!: refuse safety_identifier on providers that cannot take it unless drop_params drops it #44955)
• 674208b fix(model_prices): drop unverified output price for azure_ai/Microsoft-Decision-1 (fix(model_prices): drop unverified output price on Microsoft-Decision-1 and guard evaluation rows #45717)
• ed9c31c feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN #45678)
• bc2cc42 docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI (docs(proxy): describe /v1/decisions and /v1/systemone in Swagger UI #45838)
• 5894968 test(decisions): bill the OpenAI-format openrouter case from provider-reported cost (test(decisions): bill the OpenAI-format openrouter case from provider-reported cost #45853)
• 25d9b05 feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape #45846)

Skipped on purpose

• feat(ui): let admins find, call, and test decision models from the Admin UI #45654, feat(ui): link the decision models banner to the supported providers docs and rename the System One playground to Decisions #45732, fix(ui): drop the decision models sublabel from the Add Model provider dropdown #45755: the decisions playground and its Add Model sublabel, excluded on this line
• feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider #45200: Databricks ai_decide and the auto-router decider need the Laya and Bespoke classifier work (feat: add Laya gateway and OSS classifier providers #43626, feat: add Bespoke Nimble gateway and OSS classifier support #44246, fix(auto-router): separate tuning and select chained heuristic #44928) that is not on the line
• test(decisions): add hosted_vllm /v1/decisions and provider-400 translation cases #45661: hosted_vllm translation cases need the newer tests/integration/translation runner

Adaptations on this line

• The shared HTTP handler keeps the line's _get_httpx_client name and its DecisionsIRRequest flow, with the AgentCore signing from feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN #45678 on top
• The OpenRouter Responses API half of fix(openrouter): bill Responses, Decisions and pass-through from OpenRouter usage.cost, always allow reasoning params #45640 is left out because it imports litellm_core_utils.hidden_params, which the line does not have; the decisions and pass-through billing from usage.cost is in
• The Databricks tests that feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape #45846 added to tests/unit/decisions/test_main.py are dropped with feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider #45200
• tests/unit/conftest.py gets the per-test logging worker flush from main (perf(ci): isolate and shard integration suites #44146), without which the new systemone callback test sees callbacks queued by earlier tests
• ui/litellm-dashboard/src/lib/http/schema.d.ts is regenerated with npm run gen:api, and the Strands logo is copied from main
• The decision model guardrail (feat(guardrails): backport the decision model guardrail to rc/1.105.0 (#45663) #45894) landed on the line after this branch forked and imports supported_decisions_providers from litellm.decisions.main, which feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape #45846 replaces; as on main it now lives in litellm.decisions.call and the guardrail endpoint and its test import it from there, in the merge commit fcbb022

Testing

• tests/unit/decisions, tests/unit/systemone, tests/unit/proxy/decisions_endpoints, tests/unit/proxy/public_endpoints, the strands_decider, openrouter, azure_ai and hosted_vllm provider tests, tests/unit/proxy/auth/test_auth_utils.py and tests/unit/test_cost_map_guard.py: 146 decisions and systemone tests plus the provider suites pass on this branch
• provider_info_helpers.test.tsx: 76 passed, no type errors
• ruff reports no new findings on the touched Python files against rc/1.105.0
• The live before and after on a proxy built from this branch is the /ready follow-along video, linked from the Slack READY ping

Screenshots / Proof of Fix

Live /qa leg at this tip (fcbb022) against the merge base (495288a): two proxies booted from clean worktrees with --num_workers 2, their own Postgres and Redis, real TypeSafe, OpenRouter and Azure AI Foundry keys from the shared vault, real spend. Full command and output log in the QA record kept with the run; the flips a user sees:

# [C1] POST /v1/decisions model=typesafe/jev-1.13.0 with safety_identifier
BEFORE: HTTP 200 (param silently dropped)   x-litellm-response-cost: 1.2306e-05
AFTER:  HTTP 400 litellm.UnsupportedParamsError: typesafe does not support parameters: ['safety_identifier'] ...
# [C2] same body on a deployment with drop_params: true
BEFORE: HTTP 200   AFTER: HTTP 200

# [D] POST /v1/decisions model=openrouter/typesafe/jev-1.13 (billing from usage.cost)
BEFORE: HTTP 200 (no x-litellm-response-cost header, spend log 0)
AFTER:  HTTP 200 x-litellm-response-cost: 1.4532e-05 (spend log 1.4532e-05)

# [G1] POST /v1/systemone model=azure_ai/decision-1 (real Microsoft-Decision-1 deployment)
BEFORE: HTTP 400 Unknown Decisions provider 'azure_ai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider, openai
AFTER:  HTTP 200 {"model":"microsoft-decision-1","answers":{...}}
# [G2] same on POST /v1/decisions: BEFORE 400, AFTER 200

# [F] GET /v1/model/info azure_ai/Microsoft-Decision-1
BEFORE: input_cost_per_token null, mode null
AFTER:  input_cost_per_token 4.2e-08, output_cost_per_token 0.0, mode evaluation

# [K] POST /v1/decisions with agentcore_runtime_session_id in the body
BEFORE: HTTP 200 (forwarded)   AFTER: HTTP 401 not allowed in request body

# [L] GET /openapi.json, POST /v1/decisions
BEFORE: summary "Decisions", no description, no request example
AFTER:  summary "Ask questions about an input (OpenAI Decisions format)", description, example, schema props input/model/questions/safety_identifier

# [M] GET /guardrails/ui/add_guardrail_settings decision_model_providers (the relocated helper)
BEFORE: [perplexity, typesafe, openrouter, cloudflare, strands_decider, openai]
AFTER:  [openai, openrouter, azure_ai, perplexity, typesafe, strands_decider, cloudflare, hosted_vllm]

# Admin UI Add Model provider dropdown (Playwright, before :44105 / after :44106)
BEFORE: no TypeSafe or Strands Decider entries (GET /public/providers/fields: 0 matches)
AFTER:  "TypeSafe" and "Strands Decider" entries present with their default model placeholders

Two picks could not be exercised live and are blocked, not mocked: hosted_vllm (no vLLM server; the route now resolves the provider and fails on connect instead of 400 unknown provider) and the Strands Decider AgentCore call (no AgentCore runtime ARN credential in the shared vault; its request-body rejection is proven above)

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Type

🆕 New Feature

Caveats (if any)

Medium

• No Databricks decisions provider on this line, so databricks/... decisions requests still return the unknown provider 400
• The OpenRouter Responses API still bills from the cost map on this line, only decisions and pass-through use usage.cost

Low

• fix(decisions)!: refuse safety_identifier on providers that cannot take it unless drop_params drops it #44955 is a behavior change: a non-string safety_identifier on a provider that cannot take it is now a 400 unless drop_params drops it

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 4 committers have signed the CLA.

✅ mateo-berri
✅ yuneng-berri
❌ devin-ai-integration[bot]
❌ berriai-litellm-provider-info-sync[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit fcbb022. Configure here.}

[cursor]

State field misroutes OpenAI decisions

Medium Severity

_request_fields treats any state key in kwargs as the deprecated System One format. The proxy forwards the full body into adecisions, and OpenAIDecisionRequestBody allows extra properties, so a /v1/decisions request that also includes state is handled as System One. The OpenAI questions list then fails the Mapping adapter and raises an uncaught ValidationError instead of a 400. Mixed input and state on litellm.decisions is no longer rejected.

Additional Locations (1)

• litellm/proxy/decisions_endpoints/endpoints.py#L143-L150

 

^{Reviewed by Cursor Bugbot for commit fcbb022. Configure here.}

[devin-ai-integration]

Faithful pick of #45846. litellm/decisions/main.py is byte-identical to main, so this behavior ships there today and any fix lands on main first

[mateo-berri]

@greptileai

[mateo-berri]

@litereview greptile code-review

[litereview]

✅ greptile

Completed · Some checks not run.

Score: 4/5

The provider and routing backport looks ready apart from one SDK validation regression. I ran 201 focused tests successfully and reproduced the exception-contract failure below

• Low · Map field validation failures to BadRequestError · Reply in thread

Checks and execution details

• Passed · Repository and knowledge-base context (required): Read applicable AGENTS.md files, the supplied PR metadata and diff, and Greptile's index and model-request-execution documentation
• Passed · Focused Python tests (optional): 201 passed: decisions SDK, System One SDK, Strands AgentCore transformation, proxy decisions endpoints, OpenAI decisions transformation, OpenRouter chat transformation, and cost-map guard. Installed dependencies with uv sync --frozen --extra proxy --group dev, then installed pinned prisma==0.11.0 to resolve proxy test collection
• Failed · SDK validation regression reproducer (optional): An invalid questions list raised Pydantic ValidationError instead of LiteLLM BadRequestError; the exception-contract assertion failed
• Not run · Live provider verification (optional): No live Azure, hosted vLLM, or AgentCore calls; provider verification used the focused tests

---

View review · Reviewed fcbb02231b28

[litereview]

✅ code-review

Completed · Some checks not run.

I'm the standards reviewer for #45905. I found no hard breaches of the documented standards in the code this PR adds. There is one borderline breach of the comment rule, several smells (all judgement calls), and one low-severity correctness bug.

Standards

Documented standards

• Comment rule, borderline (AGENTS.md: comments are not allowed unless "absolutely necessary"):
  • The new class docstring on AzureAIDecisionsConfig only repeats what the class says.
  • The rewritten docstring on OpenrouterConfig.get_supported_openai_params does the same.
  • These are fine:
    • The inline_request_body docstring explains a non-obvious Swagger $ref constraint.
    • The auth_utils.py comment gives a security reason.
    • The endpoint docstrings are Swagger input, which counts as tool input.
    • The fixture docstrings cite where the data came from and the date, which AGENTS.md asks for.
• Unrelated changes in the same diff: test_custom_openapi_spec.py reformats existing @​patch blocks that have nothing to do with this change. The doubles on our own code were already there, so this isn't a new breach.

Smells (judgement calls)

• Duplicated Code:
  • send/asend in litellm/decisions/call.py are the same apart from await.
  • So are systemone/asystemone, and the handler's decisions/adecisions.
  • The next(member for member in LlmProviders if member.value == ...) lookup is copied in call.py:97 and health_check_helpers.py:41.
• Repeated Switches: get_provider_decisions_config is an eight-branch if provider == ... chain. A provider→class map would let the lookup above share it.
• Primitive Obsession / duplicated knowledge: the TypeSafe passthrough handler checks custom_llm_provider == "openrouter". That repeats what OpenRouterDecisionsConfig.provider_reported_cost already knows.
• Data Clumps / Middle Man: api_key, api_base, timeout, custom_llm_provider, extra_headers, kwargs move together through _prepare_systemone_call and _prepare_decisions_call into prepare_call. Both wrappers mostly just pass them on.
• Shotgun Surgery: adding the "asystemone" call type took matching list edits across six files, nine places in all.
• Refused Bequest: SystemOneRequestFields.validate(self, safety_identifier) never uses its argument.
• Speculative Generality: the empty EVALUATION_OUTPUT_PRICE_SOURCES is threaded through as a keyword-only parameter of guard_failures and _file_failures, and only tests use it.
• Mysterious Name: _fields_checked_before_routing actually removes safety_identifier from the request data. Something like _without_safety_identifier would say that.

Correctness

• Low: non-JSON replies from the provider get past the decisions error handling (finding below).
• Behaviour change in a backport: OpenRouter chat now offers reasoning_effort and thinking for every model. Worth confirming this was meant to ride along.

Spec

Out of scope here: I was only assigned the Standards axis.

Totals: Standards has one borderline breach of the comment rule, eight smells, and one low-severity correctness bug. The worst item is the validate_json call sitting outside the try block.

• Low · Non-JSON provider replies skip the decisions error wrapper · Reply in thread

• Low · OpenRouter provider-cost check duplicated as a string check in the passthrough handler · Reply in thread

• Low · Copy-pasted provider lookup plus an if-cascade provider registry · Reply in thread

• Low · Sync and async decisions paths are copies of each other · Reply in thread

• Low · Docstrings that just restate the code · Reply in thread

Checks and execution details

• Passed · Read standards sources (required): Read AGENTS.md, tests/AGENTS.md, tests/unit/AGENTS.md, tests/integration/AGENTS.md and ui/litellm-dashboard/AGENTS.md, and grepped CONTRIBUTING.md and ARCHITECTURE.md for rules.
• Passed · Manual diff review against standards and smell baseline (required): Reviewed the non-generated source and test hunks in /tmp/pr.diff and checked suspected bugs against the checkout. Skimmed schema.d.ts and the snapshot JSON.
• Not run · Run unit tests (optional): This was a standards review done by reading code; I didn't run any tests.

---

View review · Reviewed fcbb02231b28

[litereview]

Low · Non-JSON provider replies skip the decisions error wrapper

_PAYLOAD_ADAPTER.validate_json(raw_response.content) runs before the try block. A 200 response with a non-JSON body (an HTML gateway page, say) raises a raw pydantic ValidationError instead of the BaseLLMException(status_code=500, "...returned an unexpected response") the method builds for bad payloads. The raw error then goes to litellm.exception_type through _map_upstream_exception (litellm/decisions/call.py:286). Moving the parse inside the try fixes it.

payload: Final[object] = _PAYLOAD_ADAPTER.validate_json(raw_response.content)
try:
return self.parse_response(payload, request)
except ValidationError as error:

Review by code-review · Reviewed fcbb02231b28

[devin-ai-integration]

Faithful pick, base_llm/decisions/transformation.py is byte-identical to main. Moving validate_json inside the try lands on main first, then a follow-up pick

[litereview]

Low · OpenRouter provider-cost check duplicated as a string check in the passthrough handler

Judgement call (Primitive Obsession / Duplicated Code). Using provider-reported cost is already modelled by BaseDecisionsConfig.provider_reported_cost, which OpenRouterDecisionsConfig overrides. Here the same rule is hard-coded as custom_llm_provider == "openrouter", so the two copies can drift apart.

usage.cost
if custom_llm_provider == "openrouter" and usage.cost is not None

Review by code-review · Reviewed fcbb02231b28

[devin-ai-integration]

1.105 keeps the string check on purpose, main's provider_reported_cost path needs litellm_core_utils.hidden_params from #45640, which this line does not have

[litereview]

Low · Copy-pasted provider lookup plus an if-cascade provider registry

Judgement call (Repeated Switches / Duplicated Code). get_provider_decisions_config is an eight-branch if provider == ... chain. The string-to-LlmProviders lookup that feeds it is copied in litellm/decisions/call.py:97 and litellm/litellm_core_utils/health_check_helpers.py:41. One shared map would replace all three.

provider: Final = next((member for member in LlmProviders if member.value == custom_llm_provider), None)

Review by code-review · Reviewed fcbb02231b28

[devin-ai-integration]

Judgement call on code main carries today, plus a Databricks branch there. A provider map refactor lands on main first, this backport ships merged picks

[litereview]

Low · Sync and async decisions paths are copies of each other

Judgement call (Duplicated Code). send and asend are the same apart from await. The same is true of systemone/asystemone in litellm/systemone/main.py, decisions/adecisions in litellm/decisions/main.py, and the handler methods in llm_http_handler.py. The thin _prepare_*_call wrappers also just pass the same six-parameter group on to prepare_call (Middle Man / Data Clumps).

def send(call: DecisionsCall) -> DecisionsResponse | OpenAIDecisionResponse:
try:
response: Final = _HANDLER.decisions(

Review by code-review · Reviewed fcbb02231b28

[devin-ai-integration]

Judgement call, call.py is byte-identical to main. Deduplicating send and asend is a main refactor, out of scope for a release-line backport

[litereview]

Low · Docstrings that just restate the code

AGENTS.md says not to write comments unless they are absolutely necessary to explain complex business logic. The new class docstring here, and the rewritten OpenrouterConfig.get_supported_openai_params docstring ("Include reasoning parameters regardless of model-map capabilities."), only repeat what the code already says.

"""Microsoft Foundry decision models (Microsoft-Decision-1), where model is the deployment name"""

Review by code-review · Reviewed fcbb02231b28

[devin-ai-integration]

Faithful pick, the docstring is identical on main. Trimming it belongs on main first, this backport only carries merged code

[litereview]

Low · Map field validation failures to BadRequestError

Calling adecisions(model="typesafe/jev-1.13", input="review", questions=[7], api_key="test-key") now raises a raw Pydantic ValidationError without a status code. _request_fields validates before entering prepare_call's error mapping, so SDK callers catching litellm.BadRequestError miss this failure. The previous implementation converted request-validation failures to BadRequestError. Move these validations inside that mapping or catch and translate them here

A local reproducer failed its isinstance(error, litellm.BadRequestError) assertion; the traceback points to _OPENAI_QUESTIONS_ADAPTER.validate_python on line 36

Review by greptile · Reviewed fcbb02231b28

[devin-ai-integration]

Faithful pick of #45846, litellm/decisions/main.py is byte-identical to main, so this behavior ships there today and any fix lands on main first

[mateo-berri]

LGTM