feat(decisions): backport Microsoft-Decision-1 and the decision models Admin UI to rc/1.106.0 (#45639, #45673, #45654, #45717)

[mateo-berri]

TLDR

This backports the decisions launch onto rc/1.106.0 as a key feature (# key-feature: decisions), so it ships in the next 1.106.0 rc

Problem this solves:

• The rc cannot call Microsoft-Decision-1 on /v1/systemone or /v1/decisions
• The rc's bundled cost map has no price for Microsoft-Decision-1
• Admins on the rc cannot find, call, or test decision models in the Admin UI

How it solves it:

• feat(azure): add azure_ai/Microsoft-Decision-1 #45639 adds the azure_ai/Microsoft-Decision-1 cost map row
• fix(model_prices): drop unverified output price on Microsoft-Decision-1 and guard evaluation rows #45717 drops that row's unverified output price, as main did
• feat(azure_ai): support Microsoft-Decision-1 on the decisions API #45673 adds azure_ai deployments to the decisions API
• feat(ui): let admins find, call, and test decision models from the Admin UI #45654 adds the decision models banner, Provider search, and playground model picker

Intentional product change (from #45654): the cost optimization feedback banner on Models + Endpoints is replaced by the decision models announcement, so admins lose that page's link to the feedback form. The announcement has its own dismissal, so admins who closed the old banner still see it once

User Flow

Before

A proxy admin on rc/1.106.0 with a Microsoft-Decision-1 deployment on Azure AI Foundry cannot call it, price it, find it, or test it

1. They add the deployment as azure_ai/decision-1 and send POST https://litellm-domain/v1/systemone. It answers 400 "Unknown Decisions provider 'azure_ai'"
2. GET https://litellm-domain/v1/model/info shows no input or output price for it
3. On https://litellm-domain/ui/models-and-endpoints the banner asks for cost optimization feedback, and nothing mentions decision models
4. In Add Model, typing decision into Provider shows "No providers found"
5. https://litellm-domain/ui/playground?tab=system-one on /v1/systemone has no Model picker, and Send fails with "Invalid model name passed in model=your-decision-model"

After

The same admin gets calibrated answers from Microsoft-Decision-1, billed for input tokens at its Azure price, and can find and test it in the Admin UI

1. The same POST https://litellm-domain/v1/systemone answers 200 with noul and choice answers and an input-only x-litellm-response-cost
2. GET https://litellm-domain/v1/model/info shows $0.042 per 1M input tokens and $0 for output, and the Deployed Models table shows IN $0.04 and OUT $0.00
3. The banner says "Decision models are now supported" with "Add a decision model" and "Try decision models" buttons
4. Typing decision into Provider lists Azure AI Foundry (Studio) with "Decision models: Microsoft-Decision-1", plus the other decision providers
5. The playground on /v1/systemone has a Model picker with microsoft-decision-1, and Send shows calibrated probabilities

Picks

Cherry-picked from main in merge order, one commit per PR, with the original author kept

• feat(azure): add azure_ai/Microsoft-Decision-1 #45639 feat(azure): add azure_ai/Microsoft-Decision-1 (de6a071)
• feat(azure_ai): support Microsoft-Decision-1 on the decisions API #45673 feat(azure_ai): support Microsoft-Decision-1 on the decisions API (2fcc780)
• feat(ui): let admins find, call, and test decision models from the Admin UI #45654 feat(ui): let admins find, call, and test decision models from the Admin UI (e2276ac)
• fix(model_prices): drop unverified output price on Microsoft-Decision-1 and guard evaluation rows #45717 fix(model_prices): drop unverified output price for azure_ai/Microsoft-Decision-1 (674208b)

Every added and removed line in each pick is identical to its source commit (2 files +14, 9 files +216, 29 files +1135 -162, and 4 files +114 -8). #45717 applied clean. The other conflicts were only in context lines that main has and the rc does not, and each was resolved by keeping just the pick's own lines

• feat(azure): add azure_ai/Microsoft-Decision-1 #45639 conflicted in both cost map JSON files next to the twelvelabs.pegasus-1-5-v1:0 row, which is on main but not the rc. Only the azure_ai/Microsoft-Decision-1 row went in
• feat(azure_ai): support Microsoft-Decision-1 on the decisions API #45673 conflicted in three test files next to entries from main-only PRs. test_decisions_wire.py got only the azure_ai row (the Databricks row is feat(decisions): add Databricks ai_decide as a /v1/decisions provider and auto-router decider #45200), proxy_config.yaml got only the azure_ai/decision-1 model (the hosted_vllm entry is feat(decisions): add hosted_vllm provider #45501), and tests/unit/decisions/test_main.py got only the azure_ai tests (the OpenRouter cost test is fix(openrouter): bill Responses, Decisions and pass-through from OpenRouter usage.cost, always allow reasoning params #45640)
• feat(ui): let admins find, call, and test decision models from the Admin UI #45654 conflicted in the imports of AddModelForm.integration.test.tsx. Only the EMPTY_DECISION_CATALOG import went in, since the other main-side imports belong to tests the rc does not have

#45717 merged after this backport started and is in because it fixes the price row #45639 adds. Without it the rc would bill output tokens at a price main already dropped

No hidden prerequisite was needed. The picked tests pass on the line without #44955, #45200, #45501, or #45640, which stay off the rc

Tests on the backport tip

• tests/unit/decisions/test_main.py and tests/unit/test_cost_map_guard.py pass (80 tests, the new azure_ai and evaluation-row ones included)
• ci_cd/cost_map_guard.py passes against the rc base
• The lazy import and cost map tests pass (46 passed, 1 skipped)
• The dashboard vitest files the picks touch pass (11 files, 145 tests), and npm run typecheck exits 0

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two local proxies with Postgres, one at the rc tip and one at this PR's tip, each run with LITELLM_LOCAL_MODEL_COST_MAP=True so the bundled cost map is the one priced. Both call the same real Microsoft-Decision-1 GlobalStandard deployment (decision-1) on Azure AI Foundry. The Admin UI ran through each side's dashboard dev server in headless Chromium at 1440x900, logged in as the proxy admin

model_list:
  - model_name: microsoft-decision-1
    litellm_params:
      model: azure_ai/decision-1
      api_base: https://lit6549-audio-qa.services.ai.azure.com
      api_key: os.environ/AZURE_AI_API_KEY
    model_info:
      mode: evaluation
      base_model: azure_ai/Microsoft-Decision-1

systemone.json

{"model":"microsoft-decision-1","state":"Ticket (billing): The export job hangs at 99% and never finishes","questions":{"defect":{"type":"noul","instructions":"Is this a defect?"},"severity":{"type":"choice","instructions":"How severe is it?","criteria":{"low":"cosmetic","high":"blocks users"}}}}

Before (bbcc985, rc/1.106.0 tip)

System One call (#45673)

1. curl -sS -i -X POST http://localhost:$PORT/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @systemone.json
2. Output

HTTP/1.1 400 Bad Request
x-litellm-version: 1.106.0
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'azure_ai'. Supported providers: openai, openrouter, perplexity, typesafe, strands_decider, cloudflare\n\nLiteLLM: model group 'microsoft-decision-1' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

Price (#45639)

1. curl -sS http://localhost:$PORT/v1/model/info -H "Authorization: Bearer $LITELLM_MASTER_KEY"
2. The microsoft-decision-1 entry has "key": "azure_ai/decision-1", "input_cost_per_token": null, and "output_cost_per_token": null

Admin UI (#45654)

1. Open http://localhost:$DEV/ui/models-and-endpoints. The banner asks for cost optimization feedback (first screenshot under User Flow Before)
2. Click Add Model and type decision into Provider. It shows "No providers found" (second screenshot)
3. Open http://localhost:$DEV/ui/playground?tab=system-one, pick System One /v1/systemone as the Endpoint, and click Send. There is no Model picker, and the response panel says "System One request failed /decisions: Invalid model name passed in model=your-decision-model" (third screenshot)

After (8b631ac)

System One call (#45673)

1. The same curl
2. Output

HTTP/1.1 200 OK
x-litellm-model-api-base: https://lit6549-audio-qa.services.ai.azure.com
x-litellm-version: 1.106.0
x-litellm-response-cost: 2.73e-06
x-litellm-response-cost-input: 2.73e-06
x-litellm-response-cost-output: 0.0
{"model":"microsoft-decision-1","answers":{"defect":{"type":"noul","noul":0.982013764002278},"severity":{"type":"choice","choice":"high","confidence":0.9866142978108987,"probabilities":{"high":0.9933071489054494,"low":0.006692851094550702}}},"usage":{"input_tokens":65,"output_tokens":2}}

The cost is 65 input tokens at $0.042 per 1M, the row #45639 adds, and output is free per #45717

Price (#45639, #45717)

1. The same /v1/model/info curl
2. The entry now has "key": "azure_ai/Microsoft-Decision-1", "input_cost_per_token": 4.2e-08, and "output_cost_per_token": 0.0

Admin UI (#45654)

1. Open http://localhost:$DEV/ui/models-and-endpoints. The banner says "Decision models are now supported", and the deployment's Costs column shows IN $0.04 and OUT $0.00 (first screenshot under User Flow After)
2. Click Add Model and type decision into Provider. Azure AI Foundry (Studio) is listed with "Decision models: Microsoft-Decision-1", followed by Cloudflare, OpenAI, OpenRouter, Perplexity, Strands Decider, and TypeSafe (second screenshot)
3. Open http://localhost:$DEV/ui/playground?tab=system-one, pick System One /v1/systemone, and click Send. The Model picker shows microsoft-decision-1, and the panel shows calibrated probabilities from it, with 215 input and 3 output tokens (third screenshot)

Type

🆕 New Feature

Caveats (if any)

Copied from the source PRs and re-read against rc/1.106.0. #45639 and #45673 listed none

Low

• Chat-mode deciders like gpt-6-luna are typed into the JSON, not picked
• The AI Hub decision example uses requests, not the OpenAI SDK
• Add Model still leaves Mode empty for decision models, as before
• Switching to a typed key shows TypeSafe until that key is entered
• Add Model lists Microsoft-Decision-1 under Azure AI Foundry, and its /v1/decisions support (feat(azure_ai): support Microsoft-Decision-1 on the decisions API #45673) is in this backport too
• If Azure later documents output billing, the key goes into EVALUATION_OUTPUT_PRICE_SOURCES with the quoted source before the price can come back (fix(model_prices): drop unverified output price on Microsoft-Decision-1 and guard evaluation rows #45717)
• The guard workflow runs on pull_request_target, so this PR is checked with the rc's guard code and the new rule applies to later rc PRs (fix(model_prices): drop unverified output price on Microsoft-Decision-1 and guard evaluation rows #45717)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 3 committers have signed the CLA.

❌ devin-ai-integration[bot]
❌ moyai-devin-berriai[bot]
❌ berriai-litellm-provider-info-sync[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no new blocking issue established.

Summary

Backports Azure AI decision calls, model pricing, and Admin UI discovery and testing. The latest changes set Microsoft decision output pricing to zero and add a check for evaluation models with output prices.

• Azure AI Foundry deployments can answer LiteLLM decision requests.
• Microsoft-Decision-1 requests now have a bundled input price.
• Admins can find decision models and see how to call them.
• The System One playground can pick a decision model from the active key.

_{Reviews (2) · Last reviewed commit: "fix(model_prices): drop unverified outpu..." · Reviewed by Greptile}

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8b631ac. Configure here.}