feat(decisions): add OpenAI as a Decisions provider behind a shared decisions format

[mateo-berri]

TLDR

Problem this solves:

• OpenAI's Decisions API (gpt-6-luna) was not a Decisions provider
• openai/gpt-6-luna deployments got a 400 on both Decisions routes
• /v1/decisions rejected input_image parts with a 400 for every provider

How it solves it:

• Both routes now convert to one shared decisions format inside the proxy
• Each provider translates that shared format to and from its own wire format
• New openai provider calls https://api.openai.com/v1/decisions, passing OpenAI-format input and questions through field for field
• /v1/decisions to OpenAI keeps images, multi-message input, safety_identifier, names and refusals
• Jev-compatible providers get System One bodies exactly as before, and images there return a clear 400
• Cached and cache-write input tokens are billed at their own rates
• The gpt-6-luna price row now lists /v1/decisions
• The openai provider reads litellm.api_key, litellm.openai_key, litellm.api_base, OPENAI_BASE_URL and OPENAI_API_BASE in the same order as other OpenAI calls
• System One choice or score questions without instructions get a short default one for OpenAI, and single-option questions get a clear 400 before any upstream call
• Built on main's BaseDecisionsConfig and shared HTTP handler dispatch from refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler #45130: OpenAIDecisionsConfig is registered in ProviderConfigManager, and every config translates the shared format through transform_decisions_request and parse_response

Intentional product change: /v1/systemone responses only carry answers for the questions the caller asked; extra answer keys a provider returns are dropped, while extra fields inside each answer are kept

Stacked on #45184, so until that merges the diff also shows its commits. The new work is the commits after 77872f6

User Flow

Before: a proxy admin who adds openai/gpt-6-luna as a decider gets a 400 on every Decisions call

1. They add model: openai/gpt-6-luna as luna next to their existing typesafe/jev-latest as jev
2. A developer sends POST http://localhost:4000/v1/decisions with "model": "luna", input messages and a questions list
3. They get 400 Unknown Decisions provider 'openai', the same on POST http://localhost:4000/v1/systemone
4. Adding an input_image part fails validation with a 400, even for jev

After: both models answer on both routes, and images reach OpenAI

1. They add model: openai/gpt-6-luna as luna next to their existing typesafe/jev-latest as jev
2. A developer sends the same POST http://localhost:4000/v1/decisions body
3. They get 200 with an answers list carrying their question names, and /v1/systemone returns an answers map keyed by name
4. An input_image part gets a real answer from luna, while jev returns 400 saying it accepts text input only
5. Every 200 carries a non-zero x-litellm-response-cost header

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

Proxy started with python -m litellm.proxy.proxy_cli --config both_config.yaml --port 4000, calling OpenAI and Typesafe for real

model_list:
  - model_name: luna
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY
  - model_name: jev
    litellm_params:
      model: typesafe/jev-latest
      api_key: os.environ/TYPESAFE_API_KEY

general_settings:
  master_key: os.environ/DECISIONS_PROOF_MASTER_KEY

luna_systemone_body.json (jev_systemone_body.json is the same with "model": "jev"):

{
  "model": "luna",
  "state": "Customer wrote: I was charged twice for my order #4411 and want one of the charges refunded today.",
  "questions": {
    "is_refund_request": {
      "type": "noul",
      "instructions": "Is the customer asking for a refund?"
    },
    "urgency": {
      "type": "choice",
      "instructions": "How urgent is this?",
      "criteria": {
        "low": "can wait a week",
        "high": "needs action today"
      }
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated is the customer?",
      "criteria": [
        "calm",
        "mildly annoyed",
        "angry"
      ]
    }
  }
}

luna_openai_body.json (jev_openai_body.json is the same with "model": "jev"):

{
  "model": "luna",
  "input": [
    {
      "role": "user",
      "type": "message",
      "content": [
        {
          "type": "input_text",
          "text": "Customer wrote: I was charged twice for my order #4411 and want one of the charges refunded today."
        }
      ]
    }
  ],
  "questions": [
    {
      "type": "predicate",
      "name": "is_refund_request",
      "instructions": "Is the customer asking for a refund?"
    },
    {
      "type": "choice",
      "name": "needs_human",
      "instructions": "Should a human agent take over?",
      "choices": [
        {
          "value": true,
          "description": "billing dispute that needs a person"
        },
        {
          "value": false
        }
      ]
    },
    {
      "type": "score",
      "name": "frustration",
      "instructions": "How frustrated is the customer?",
      "levels": [
        {
          "label": "calm"
        },
        {
          "label": "mildly annoyed"
        },
        {
          "label": "angry",
          "description": "uses hostile language"
        }
      ]
    }
  ]
}

luna_image_body.json (jev_image_body.json is the same with "model": "jev"):

{
  "model": "luna",
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "Here is the swatch the customer uploaded."
        },
        {
          "type": "input_image",
          "image_url": "data:image/png;base64,iVBORw0KGgoAAAANSU...(64x64 solid red PNG)",
          "detail": "low"
        }
      ]
    }
  ],
  "questions": [
    {
      "type": "predicate",
      "name": "is_red",
      "instructions": "Is the swatch mostly red?"
    },
    {
      "type": "choice",
      "name": "color",
      "instructions": "Which color dominates the swatch?",
      "choices": [
        {
          "value": "red"
        },
        {
          "value": "green"
        },
        {
          "value": "blue"
        }
      ]
    }
  ],
  "safety_identifier": "proof-user-1"
}

Before (77872f6)

luna, System One format at /v1/systemone

1. curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_systemone_body.json
2. Output:

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

luna, OpenAI format at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_openai_body.json
2. Output:

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

luna, OpenAI format with an image at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_image_body.json
2. Output:

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Invalid Decisions request: 6 validation errors for OpenAIDecisionRequestBody\ninput.str\n  Input should be a valid string [type=string_type, input_value=[{'type': 'message', 'rol...==', 'detail': 'low'}]}], input_t ... (remaining pydantic validation errors trimmed)

jev, System One format at /v1/systemone

1. curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_systemone_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 1.6422e-05
{"model":"jev-1.13.0","answers":{"is_refund_request":{"type":"noul","noul":0.99},"urgency":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"low":0.0,"high":1.0}},"frustration":{"type":"score","score":0.95,"confidence":0.86,"legend":{"0":"calm","1":"mildly annoyed","2":"angry"},"probabilities":{"0":0.07,"1":0.91,"2":0.02}}},"usage":{"input_tokens":391,"output_tokens":67}}

jev, OpenAI format at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_openai_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 1.6422e-05
{"model":"jev-1.13.0","answers":[{"type":"predicate","name":"is_refund_request","probability":0.99},{"type":"choice","name":"needs_human","choice":true,"probabilities":[{"value":true,"probability":0.65},{"value":false,"probability":0.35}],"confidence":0.3},{"type":"score","name":"frustration","score":0.84,"probabilities":[{"value":0,"label":"calm","probability":0.16},{"value":1,"label":"mildly annoyed","probability":0.84},{"value":2,"label":"angry","probability":0.0}],"confidence":0.76}],"usage":{"input_tokens":391,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":61,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":452}}

jev, OpenAI format with an image at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_image_body.json
2. Output:

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Invalid Decisions request: 6 validation errors for OpenAIDecisionRequestBody\ninput.str\n  Input should be a valid string [type=string_type, input_value=[{'type': 'message', 'rol...==', 'detail': 'low'}]}], input_t ... (remaining pydantic validation errors trimmed)

After (39012f1)

luna, System One format at /v1/systemone

1. curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_systemone_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 3.9699999999999996e-05
{"model":"gpt-6-luna","answers":{"is_refund_request":{"type":"noul","noul":1.0},"urgency":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"low":0.0,"high":1.0}},"frustration":{"type":"score","score":0.96,"confidence":0.76,"legend":{"0":"calm","1":"mildly annoyed","2":"angry"},"probabilities":{"0":0.1,"1":0.84,"2":0.06}}},"usage":{"input_tokens":397,"output_tokens":0}}

luna, OpenAI format at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_openai_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 4.01e-05
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_refund_request","probability":1.0},{"type":"choice","name":"needs_human","choice":true,"probabilities":[{"value":true,"probability":0.99},{"value":false,"probability":0.01}],"confidence":0.98},{"type":"score","name":"frustration","score":0.74,"probabilities":[{"value":0,"label":"calm","probability":0.26},{"value":1,"label":"mildly annoyed","probability":0.74},{"value":2,"label":"angry","probability":0.0}],"confidence":0.61}],"usage":{"input_tokens":401,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":401}}

luna, OpenAI format with an image at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @luna_image_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 2.77e-05
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_red","probability":1.0},{"type":"choice","name":"color","choice":"red","probabilities":[{"value":"red","probability":1.0},{"value":"green","probability":0.0},{"value":"blue","probability":0.0}],"confidence":1.0}],"usage":{"input_tokens":277,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":277}}

jev, System One format at /v1/systemone

1. curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_systemone_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 1.6422e-05
{"model":"jev-1.13.0","answers":{"is_refund_request":{"type":"noul","noul":0.99},"urgency":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"low":0.0,"high":1.0}},"frustration":{"type":"score","score":0.95,"confidence":0.85,"legend":{"0":"calm","1":"mildly annoyed","2":"angry"},"probabilities":{"0":0.08,"1":0.9,"2":0.02}}},"usage":{"input_tokens":391,"output_tokens":67}}

jev, OpenAI format at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_openai_body.json
2. Output:

HTTP/1.1 200 OK
x-litellm-response-cost: 1.6422e-05
{"model":"jev-1.13.0","answers":[{"type":"predicate","name":"is_refund_request","probability":0.99},{"type":"choice","name":"needs_human","choice":true,"probabilities":[{"value":true,"probability":0.64},{"value":false,"probability":0.36}],"confidence":0.28},{"type":"score","name":"frustration","score":0.84,"probabilities":[{"value":0,"label":"calm","probability":0.16},{"value":1,"label":"mildly annoyed","probability":0.84},{"value":2,"label":"angry","probability":0.0}],"confidence":0.76}],"usage":{"input_tokens":391,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":68,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":459}}

jev, OpenAI format with an image at /v1/decisions

1. curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @jev_image_body.json
2. Output:

HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Decisions provider 'typesafe' cannot serve this request: input_image content parts are not supported because System One providers accept text input only\n\nLiteLLM: model group 'jev' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

Custom pricing (39012f1)

A second proxy with per-token prices set on the deployment, called twice with a ~1.8k-token input (cost_openai_body.json) to see whether OpenAI reports cached tokens

model_list:
  - model_name: luna-priced
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY
      input_cost_per_token: 0.000001
      output_cost_per_token: 0.000002
      cache_read_input_token_cost: 0.0000001
      cache_creation_input_token_cost: 0.00000125

general_settings:
  master_key: os.environ/DECISIONS_PROOF_MASTER_KEY

=== call 1: curl -i localhost:4012/v1/decisions -d @cost_openai_body.json
HTTP/1.1 200 OK
x-litellm-response-cost: 0.001854
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_refund_request","probability":1.0}],"usage":{"input_tokens":1854,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":1854}}
expected (input-cached-write)*1e-6 + cached*1e-7 + write*1.25e-6 + output*2e-6 = 0.001854000; proxy billed 0.001854000; match=True

=== call 2: curl -i localhost:4012/v1/decisions -d @cost_openai_body.json
HTTP/1.1 200 OK
x-litellm-response-cost: 0.001854
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_refund_request","probability":1.0}],"usage":{"input_tokens":1854,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":1854}}
expected (input-cached-write)*1e-6 + cached*1e-7 + write*1.25e-6 + output*2e-6 = 0.001854000; proxy billed 0.001854000; match=True

Billing matches the configured rates. OpenAI and Typesafe both returned 0 cached tokens on every repeat (5 and 3 identical calls), so the cached-token path is covered by test_custom_token_pricing_bills_cached_decisions_input_tokens_once, which fails on 85f9121 and passes here

QA notes from the run:

• A choice value of 1 reaches OpenAI as true
• Choices true and "true" together are rejected for OpenAI too
• Duplicate names: luna returns OpenAI's 400, jev returns 200
• OpenAI reports output_tokens as 0 for luna
• /v1/systemone usage has no total_tokens
• One-option System One choice to luna: OpenAI 400 names choices
• Jev sent no answer extras, so unit test covers fix

OpenAI key from litellm_settings (e465919)

The proxy runs with no OPENAI_API_KEY in its environment and the key set only through litellm_settings.openai_key, which other OpenAI calls already honor

model_list:
  - model_name: luna
    litellm_params:
      model: openai/gpt-6-luna

litellm_settings:
  openai_key: os.environ/DECISIONS_PROOF_OPENAI_KEY

general_settings:
  master_key: os.environ/DECISIONS_PROOF_MASTER_KEY

Before, at 39012f1, the provider never looked at litellm.openai_key

$ curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @body.json
HTTP/1.1 401 Unauthorized
x-litellm-response-cost: 0
{"error":{"message":"litellm.AuthenticationError: Missing API key for Decisions provider 'openai'\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"authentication_error","param":null,"code":"401"}}

After, at e465919, the same request reaches OpenAI and gets a real answer

$ curl -sS -i localhost:4000/v1/decisions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @body.json
HTTP/1.1 200 OK
x-litellm-response-cost: 1.68e-05
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_defect","probability":0.52}],"usage":{"input_tokens":168,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":168}}

System One questions without instructions (e4081f7)

Same config as above. Before, at a2951b3, OpenAI rejected the translated request for the criteria-only choice and score questions, and the single-option question cost an upstream round trip

$ cat noinst.json
{"model": "luna", "state": "My new phone arrived with a cracked screen and I want my money back.", "questions": {"action": {"type": "choice", "criteria": {"refund": "Customer wants money back", "replace": "Customer wants a new unit", "escalate": null}}, "severity": {"type": "score", "criteria": ["cosmetic", "partly usable", "unusable"]}}}
$ curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @noinst.json
HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: OpenAIException - {\"error\":{\"message\":\"Missing required parameter: 'questions[0].instructions'.\",\"type\":\"invalid_request_error\",\"param\":\"questions[0].instructions\",\"code\":\"missing_required_parameter\"}}\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

$ cat oneopt.json
{"model": "luna", "state": "My new phone arrived with a cracked screen.", "questions": {"action": {"type": "choice", "instructions": "What should we do?", "criteria": {"refund": "Customer wants money back"}}}}
$ curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @oneopt.json
HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: OpenAIException - {\"error\":{\"message\":\"Invalid 'questions[0].choices': array too short. Expected an array with minimum length 2, but got an array with length 1 instead.\",\"type\":\"invalid_request_error\",\"param\":\"questions[0].choices\",\"code\":\"array_below_min_length\"}}\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

After, at e4081f7, the criteria-only questions get real answers from OpenAI and the single-option question gets a litellm 400 that says why

$ cat noinst.json
{"model": "luna", "state": "My new phone arrived with a cracked screen and I want my money back.", "questions": {"action": {"type": "choice", "criteria": {"refund": "Customer wants money back", "replace": "Customer wants a new unit", "escalate": null}}, "severity": {"type": "score", "criteria": ["cosmetic", "partly usable", "unusable"]}}}
$ curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @noinst.json
HTTP/1.1 200 OK
x-litellm-response-cost: 2.48e-05
{"model":"gpt-6-luna","answers":{"action":{"type":"choice","choice":"refund","confidence":1.0,"probabilities":{"refund":1.0,"replace":0.0,"escalate":0.0}},"severity":{"type":"score","score":1.17,"confidence":0.0,"legend":{"0":"cosmetic","1":"partly usable","2":"unusable"},"probabilities":{"0":0.29,"1":0.25,"2":0.46}}},"usage":{"input_tokens":248,"output_tokens":0}}

$ cat oneopt.json
{"model": "luna", "state": "My new phone arrived with a cracked screen.", "questions": {"action": {"type": "choice", "instructions": "What should we do?", "criteria": {"refund": "Customer wants money back"}}}}
$ curl -sS -i localhost:4000/v1/systemone -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d @oneopt.json
HTTP/1.1 400 Bad Request
x-litellm-response-cost: 0
{"error":{"message":"litellm.BadRequestError: Decisions provider 'openai' cannot serve this request: OpenAI needs at least 2 choices or levels on every choice or score question\n\nLiteLLM: model group 'luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}

After merging main's provider config dispatch (48f1a9c)

Same luna config and bodies as the After runs above, on the tip that ports this PR onto #45130. Both routes still reach OpenAI and bill, and the litellm_settings key and single-option runs from the previous section were repeated here with the same output

=== systemone
HTTP/1.1 200 OK
x-litellm-response-cost: 3.9699999999999996e-05
{"model":"gpt-6-luna","answers":{"is_refund_request":{"type":"noul","noul":1.0},"urgency":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"low":0.0,"high":1.0}},"frustration":{"type":"score","score":0.96,"confidence":0.76,"legend":{"0":"calm","1":"mildly annoyed","2":"angry"},"probabilities":{"0":0.1,"1":0.84,"2":0.06}}},"usage":{"input_tokens":397,"output_tokens":0}}

=== decisions
HTTP/1.1 200 OK
x-litellm-response-cost: 4.01e-05
{"model":"gpt-6-luna","answers":[{"type":"predicate","name":"is_refund_request","probability":1.0},{"type":"choice","name":"needs_human","choice":true,"probabilities":[{"value":true,"probability":0.99},{"value":false,"probability":0.01}],"confidence":0.98},{"type":"score","name":"frustration","score":0.74,"probabilities":[{"value":0,"label":"calm","probability":0.26},{"value":1,"label":"mildly annoyed","probability":0.74},{"value":2,"label":"angry","probability":0.0}],"confidence":0.61}],"usage":{"input_tokens":401,"input_tokens_details":{"cached_tokens":0,"cache_write_tokens":0},"output_tokens":0,"output_tokens_details":{"reasoning_tokens":0},"total_tokens":401}}

Type

New Feature

Caveats (if any)

Medium

• OpenAI has not published Decisions API pricing yet
  • Cost uses gpt-6-luna token rates until it does
• System One questions with a single choice or level get a 400 on openai deployments, since OpenAI needs at least 2

Low

• Noul criteria are folded into the predicate's instructions as "Answer true when: ..." lines
• Choice or score questions without instructions are sent to OpenAI with "Which choice best fits the input?" or "Which level best fits the input?"
• Score levels use the criterion text as the label, JSON-encoded when it is not a string
• Jev response extras like latency_ms only show up on /v1/systemone, not in OpenAI-format answers
• A choice value of 1 is coerced to true by request validation (from feat(decisions): serve System One format at /v1/systemone and OpenAI format at /v1/decisions #45184), where OpenAI itself would return 400
• Choices true and "true" in one question get a 400 from the System One key check even when routed to OpenAI, which accepts them
• Backported in feat(decisions): backport /v1/systemone, OpenAI-format /v1/decisions and the openai Decisions provider to rc/1.105.0 (#44236, #45184, #45214) #45189 and feat(decisions): backport /v1/systemone, OpenAI-format /v1/decisions and the OpenAI Decisions provider to stable/1.104.x for v1.104.2 (#44236, #45184, #45214) #45190

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches request routing, cross-format translation, and billing for a new provider API; mis-translation or usage mapping could cause wrong answers or spend, but behavior is heavily tested.

Overview
Adds OpenAI as a Decisions provider and splits the proxy into two request shapes: /v1/systemone (state + keyed noul/choice/score questions) and /v1/decisions (OpenAI-style input, question list, safety_identifier, including images).

decisions / adecisions now validate either format, reject mixing state and input, normalize through a shared IR, let each provider build the upstream body via transform_decisions_request, then map the IR response back to System One or OpenAI answers. OpenAIDecisionsConfig targets https://api.openai.com/v1/decisions; Jev-style providers still use /v1/systemone wire bodies. Unsupported cases (e.g. images to text-only providers, single-option choice/score for OpenAI) fail with 400 before HTTP.

Cost and logging understand OpenAIDecisionResponse usage (cached / cache-write input tokens). gpt-6-luna pricing lists /v1/decisions. Gateway allowlists, lazy routes, auth, integration tests, and the playground default shift legacy Jev traffic to /v1/systemone; /v1/decisions stays available for pass-through when configured.

^{Reviewed by Cursor Bugbot for commit c19ada6. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds OpenAI as a decisions provider with shared format transformation.

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains

What we checked:

• Frozen responses keep guardrail results: attach_guardrail_information returns a copy with one added field. Both empty and populated results use this path

Summary

This PR adds OpenAI as a Decisions provider and gives both public request formats a shared provider path. Since the previous review, it allows extra fields on OpenAIDecisionResponse so the proxy can attach requested guardrail results

• Adds a regression test for include_guardrail_response on both routes
• The previous provider-helper location finding is fixed. The helpers now live under llms/
• The previous answer-extra finding is fixed. The shared format preserves extras inside requested answers
• mateo-berri accepted the route change because the old behavior shipped only in a development release. Greptile withdrew that compatibility finding
• The PR explicitly accepts dropping unasked answer keys, interim OpenAI pricing, single-option rejection, numeric-to-boolean coercion, and boolean/string choice collisions. These are not new findings

_{Reviews (10) · Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." · Reviewed by Greptile}

[mateo-berri]

@greptileai please re-review now that the OpenAI wire translation lives under llms/openai as you suggested

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_openai_decisions_provider (7ac5bde) with main (8fc31b1)}

[codecov]

Codecov Report

❌ Patch coverage is 94.80198% with 21 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/base_llm/decisions/transformation.py	92.38%	8 Missing ⚠️
litellm/llms/openai/decisions/transformation.py	93.69%	7 Missing ⚠️
litellm/decisions/main.py	91.66%	4 Missing ⚠️
litellm/cost_calculator.py	87.50%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai please re-review the latest commit, which routes both Decisions request formats through one shared format

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Answer name is required on parse
  • Defaulted OpenAI answer name to None so provider responses that omit the key on unnamed answers parse instead of raising.

Or push these changes by commenting:

@cursor push 6d895d3d5b

Preview (6d895d3d5b)

diff --git a/litellm/types/decisions.py b/litellm/types/decisions.py
--- a/litellm/types/decisions.py
+++ b/litellm/types/decisions.py
@@ -235,7 +235,7 @@
 
 class OpenAIPredicateAnswer(LiteLLMPydanticObjectBase):
     type: Literal["predicate"] = "predicate"
-    name: str | None
+    name: str | None = None
     probability: float
 
     model_config = ConfigDict(frozen=True)
@@ -250,7 +250,7 @@
 
 class OpenAIChoiceAnswer(LiteLLMPydanticObjectBase):
     type: Literal["choice"] = "choice"
-    name: str | None
+    name: str | None = None
     choice: str | bool
     probabilities: tuple[OpenAIChoiceProbability, ...]
     confidence: float
@@ -268,7 +268,7 @@
 
 class OpenAIScoreAnswer(LiteLLMPydanticObjectBase):
     type: Literal["score"] = "score"
-    name: str | None
+    name: str | None = None
     score: float
     probabilities: tuple[OpenAIScoreProbability, ...]
     confidence: float
@@ -278,7 +278,7 @@
 
 class OpenAIRefusalAnswer(LiteLLMPydanticObjectBase):
     type: Literal["refusal"] = "refusal"
-    name: str | None
+    name: str | None = None
 
     model_config = ConfigDict(frozen=True)
 

diff --git a/tests/unit/llms/openai/decisions/test_openai_decisions_transformation.py b/tests/unit/llms/openai/decisions/test_openai_decisions_transformation.py
--- a/tests/unit/llms/openai/decisions/test_openai_decisions_transformation.py
+++ b/tests/unit/llms/openai/decisions/test_openai_decisions_transformation.py
@@ -100,6 +100,36 @@
     assert ir_to_openai_response(parsed, ir, "requested").model_dump(mode="json") == _OPENAI_RESPONSE
 
 
+def test_parse_response_accepts_answers_that_omit_name() -> None:
+    ir: Final = _openai_ir(_OPENAI_REQUEST)
+    payload: Final = {
+        **_OPENAI_RESPONSE,
+        "answers": [
+            {"type": "predicate", "probability": 0.95},
+            {
+                "type": "choice",
+                "choice": True,
+                "probabilities": [{"value": True, "probability": 0.9}, {"value": "escalate", "probability": 0.1}],
+                "confidence": 0.8,
+            },
+            {
+                "type": "score",
+                "score": 0.7,
+                "probabilities": [
+                    {"value": 0, "label": "minor", "probability": 0.3},
+                    {"value": 1, "label": "major", "probability": 0.7},
+                ],
+                "confidence": 0.6,
+            },
+            {"type": "refusal"},
+        ],
+    }
+
+    parsed: Final = OPENAI_DECISIONS_ENDPOINT.parse_response(payload, ir)
+
+    assert ir_to_openai_response(parsed, ir, "requested").model_dump(mode="json") == _OPENAI_RESPONSE
+
+
 def test_answers_openai_did_not_return_are_refusals() -> None:
     ir: Final = _openai_ir(_OPENAI_REQUEST)
     payload: Final = {**_OPENAI_RESPONSE, "answers": [_PREDICATE_ANSWER]}

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c19ada6. Configure here.}