refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler by devin-ai-integration[bot] · Pull Request #45130 · BerriAI/litellm

devin-ai-integration · 2026-10-07T18:49:29Z

TLDR

Problem this solves:

/v1/decisions picks its provider from a hardcoded DECISIONS_ENDPOINTS dict in litellm/decisions/main.py
Adding a provider means editing that dict and the HTTP code around it
Each provider has no TranslationTestCase pinning its exact path, headers and body

How it solves it:

BaseDecisionsConfig per provider: URL, headers, request and response transforms
ProviderConfigManager.get_provider_decisions_config resolves the config from get_llm_provider
BaseLLMHTTPHandler.decisions and .adecisions do the HTTP, logging and error mapping once
One TranslationTestCase base per provider under tests/integration/translation/decisions/
Public request and response body unchanged: still the Jev System One shape
Review fix in 7933ace: _map_openrouter_exception now honors status_code_is_synthesized, so an unreachable OpenRouter raises APIConnectionError instead of a made-up 500 APIError. That also fixes OpenRouter chat and embeddings on main, which go through the same _handle_error path
Audit fix in bd81e17: _map_upstream_exception in litellm/decisions/main.py maps a connection failure (a status _handle_error synthesized and marked status_code_is_synthesized) to APIConnectionError for every decisions provider before exception_type runs. At 5276719 a Perplexity deployment that could not be reached answered litellm.InternalServerError instead of the litellm.APIConnectionError main answers, because _map_openai_exception (the mapper for every openai-compatible provider) reads the synthesized 500 as the provider's own. The alternative, teaching exception_type or _map_openai_exception to skip synthesized statuses, would change the error class of chat and embeddings connection failures for every openai-compatible provider on main, so it is not taken here
Audit cells in bd81e17: an unreachable OpenRouter deployment answers APIConnectionError on chat, streaming chat, embeddings and decisions and the SDK raises it, an unreachable decisions deployment of each of the five providers answers APIConnectionError while its sibling keeps serving (tests/unit/decisions/test_main.py::test_unreachable_upstream_maps_to_a_connection_error pins the mapper alone), and a deployment whose provider has no decisions config gets the 400 naming all five providers

PR 2 of 4 for LIT-9282, stacked on #45129. The five providers (typesafe, perplexity, openrouter, cloudflare, strands_decider) keep the same provider URL, Authorization header and wire body as before, which test_decisions_wire.py and the new base cases pin. The base cases were written from each provider's own API docs and a real captured reply, not from what main.py sends today, and they pass at this tip. PR 3 switches the public body to the OpenAI Decisions shape using the types from #45129, PR 4 adds openai/gpt-6-luna

Reading order: litellm/llms/base_llm/decisions/transformation.py, litellm/decisions/main.py, the decisions method pair in litellm/llms/custom_httpx/llm_http_handler.py, then the five 15 to 40 line provider configs under litellm/llms/<provider>/decisions/. The rest is registration rows and tests

User Flow

Before: every decisions call answers, and an OpenRouter deployment that cannot be reached reports a generic APIError where every other unreachable deployment reports APIConnectionError

They add typesafe/jev-1.13.0, openrouter/typesafe/jev-1.13, perplexity/pplx-decider-v1-27b, cloudflare/@cf/cloudflare/clef and their self-hosted strands_decider/strands-decider-2B-hobson-v19 to model_list with model_info.mode: evaluation, plus openrouter/openai/gpt-5-mini for chat, and start the proxy
They send POST http://localhost:4000/v1/decisions with "model": "typesafe/jev-1.13.0", a state string and a questions map holding a noul, a choice and a score question
They get 200 with answers keyed by question name and usage with input_tokens 368 and output_tokens 62
They repeat the same POST with each of the other four models and get the same shape back from each provider
They send POST http://localhost:4000/v1/decisions with "model": "openai/gpt-6-luna" and get 400 Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider
They point the OpenRouter chat deployment at an api_base that is down and send POST http://localhost:4000/v1/chat/completions with "model": "openrouter/openai/gpt-5-mini"; they get 500 with litellm.APIError: APIError: OpenrouterException - Cannot connect to host ..., while the same test against a TypeSafe deployment that is down says litellm.APIConnectionError: TypesafeException - Cannot connect to host ...
They point the Perplexity, OpenRouter and Cloudflare decisions deployments at an api_base that is down and send POST http://localhost:4000/v1/decisions to each; every one answers 500 litellm.APIConnectionError: <Provider>Exception - Cannot connect to host ...
They open GET http://localhost:4000/spend/logs and see one adecisions row per decisions call with the provider's own path in api_base (/v1/systemone, /alpha/decisions, /v1/decisions, /ai/run/@cf/cloudflare/clef) and the cost-map spend

After: the same calls answer the same, and the unreachable OpenRouter deployment reports APIConnectionError like every other unreachable deployment

They add typesafe/jev-1.13.0, openrouter/typesafe/jev-1.13, perplexity/pplx-decider-v1-27b, cloudflare/@cf/cloudflare/clef and their self-hosted strands_decider/strands-decider-2B-hobson-v19 to model_list with model_info.mode: evaluation, plus openrouter/openai/gpt-5-mini for chat, and start the proxy
They send POST http://localhost:4000/v1/decisions with "model": "typesafe/jev-1.13.0", a state string and a questions map holding a noul, a choice and a score question
They get 200 with answers keyed by question name and usage with input_tokens 368 and output_tokens 62
They repeat the same POST with each of the other four models and get the same shape back from each provider
They send POST http://localhost:4000/v1/decisions with "model": "openai/gpt-6-luna" and get 400 Unknown Decisions provider 'openai'. Supported providers: openrouter, perplexity, typesafe, strands_decider, cloudflare, the same five in a different order
They point the OpenRouter chat deployment at an api_base that is down and send POST http://localhost:4000/v1/chat/completions with "model": "openrouter/openai/gpt-5-mini"; they get 500 with litellm.APIConnectionError: APIConnectionError: OpenrouterException - Cannot connect to host ..., the same class the TypeSafe deployment that is down reports
They point the Perplexity, OpenRouter and Cloudflare decisions deployments at an api_base that is down and send POST http://localhost:4000/v1/decisions to each; every one still answers 500 litellm.APIConnectionError: <Provider>Exception - Cannot connect to host ...
They open GET http://localhost:4000/spend/logs and see the same adecisions rows with the same api_base paths and the same spend

Linear ticket

Resolves LIT-9282

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Two live proxies on the QA box, each --num_workers 2 with its own Postgres, one booted from the merge base and one from the tip, with a recorder in front of each real vendor. Real TypeSafe, OpenRouter, Perplexity, Cloudflare and OpenAI keys, curl as the end-user client. WIRE_<PROVIDER> is that leg's recorder, which logs each request and forwards it unchanged to the vendor; the StrandsDecider recorder has no public service behind it (it is a self-hosted decider), so its rows prove the outbound request only

Leg	Commit	Proxy
Before	`79f62db` (merge base)	127.0.0.1:58201
After	`bd81e17` (tip)	127.0.0.1:52979

Config, the same on both legs:

model_list:
  - model_name: typesafe/jev-1.13.0
    litellm_params:
      model: typesafe/jev-1.13.0
      api_key: os.environ/TYPESAFE_API_KEY
      api_base: os.environ/WIRE_TYPESAFE
    model_info:
      mode: evaluation
  - model_name: openrouter/typesafe/jev-1.13
    litellm_params:
      model: openrouter/typesafe/jev-1.13
      api_key: os.environ/OPENROUTER_API_KEY
      api_base: os.environ/WIRE_OPENROUTER
    model_info:
      mode: evaluation
  - model_name: perplexity/pplx-decider-v1-27b
    litellm_params:
      model: perplexity/pplx-decider-v1-27b
      api_key: os.environ/PERPLEXITY_API_KEY
      api_base: os.environ/WIRE_PERPLEXITY
    model_info:
      mode: evaluation
  - model_name: cloudflare/@cf/cloudflare/clef
    litellm_params:
      model: cloudflare/@cf/cloudflare/clef
      api_key: os.environ/CLOUDFLARE_API_KEY
      api_base: os.environ/WIRE_CLOUDFLARE
    model_info:
      mode: evaluation
  - model_name: strands_decider/strands-decider-2B-hobson-v19
    litellm_params:
      model: strands_decider/strands-decider-2B-hobson-v19
      api_base: os.environ/WIRE_STRANDS
    model_info:
      mode: evaluation
  - model_name: typesafe-down
    litellm_params:
      model: typesafe/jev-1.13.0
      api_key: os.environ/TYPESAFE_API_KEY
      api_base: http://127.0.0.1:9
    model_info:
      mode: evaluation
  - model_name: perplexity-down
    litellm_params:
      model: perplexity/pplx-decider-v1-27b
      api_key: os.environ/PERPLEXITY_API_KEY
      api_base: http://127.0.0.1:9
    model_info:
      mode: evaluation
  - model_name: openrouter-dec-down
    litellm_params:
      model: openrouter/typesafe/jev-1.13
      api_key: os.environ/OPENROUTER_API_KEY
      api_base: http://127.0.0.1:9
    model_info:
      mode: evaluation
  - model_name: cloudflare-down
    litellm_params:
      model: cloudflare/@cf/cloudflare/clef
      api_key: os.environ/CLOUDFLARE_API_KEY
      api_base: http://127.0.0.1:9
    model_info:
      mode: evaluation
  - model_name: typesafe-badkey
    litellm_params:
      model: typesafe/jev-1.13.0
      api_key: sk-bad-typesafe-key
      api_base: os.environ/WIRE_TYPESAFE
    model_info:
      mode: evaluation
  - model_name: openai/gpt-6-luna
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: evaluation
  - model_name: gpt-5-mini
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: openrouter-chat
    litellm_params:
      model: openrouter/openai/gpt-5-mini
      api_key: os.environ/OPENROUTER_API_KEY
  - model_name: openrouter-chat-down
    litellm_params:
      model: openrouter/openai/gpt-5-mini
      api_key: os.environ/OPENROUTER_API_KEY
      api_base: http://127.0.0.1:9
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

body_jev.json; the dec_* cells send it with .model swapped to the deployment named in the cell:

{
  "model": "typesafe/jev-1.13.0",
  "state": "Ticket (billing): The export job hangs at 99% and never finishes",
  "questions": {
    "defect": {"type": "noul", "instructions": "Is this a defect?"},
    "severity": {"type": "choice", "instructions": "How severe is it?", "criteria": {"low": "Cosmetic", "high": "Blocks the user"}},
    "confidence": {"type": "score", "instructions": "How confident are you?", "criteria": ["Not sure", "Certain"]}
  }
}

Cells, run against each leg with PROXY and KEY (the master key) set; every JSON reply is printed through jq -c and cut at 700 characters:

curl -sS -w '\nHTTP %{http_code}\n' "$PROXY/health/liveliness" -H "Authorization: Bearer $KEY"
curl -sS -w '\nHTTP %{http_code}\n' "$PROXY/health/readiness" -H "Authorization: Bearer $KEY"
curl -sS -w '\nHTTP %{http_code}\n' "$PROXY/health" -H "Authorization: Bearer $KEY"
for m in typesafe/jev-1.13.0 openrouter/typesafe/jev-1.13 perplexity/pplx-decider-v1-27b cloudflare/@cf/cloudflare/clef strands_decider/strands-decider-2B-hobson-v19 typesafe-down perplexity-down openrouter-dec-down cloudflare-down typesafe-badkey openai/gpt-6-luna nosuch/jev-1.13.0; do
  jq --arg m "$m" '.model=$m' body_jev.json | curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/decisions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d @-
done
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/decisions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"typesafe/jev-1.13.0","questions":{"q":{"type":"noul","instructions":"ok?"}}}'
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/chat/completions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"openrouter-chat","messages":[{"role":"user","content":"Reply with the single word pong"}],"max_tokens":16}'
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/chat/completions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"openrouter-chat-down","messages":[{"role":"user","content":"Reply with the single word pong"}],"max_tokens":16}'
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/chat/completions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"Reply with the single word pong"}],"max_completion_tokens":16}'
curl -sS -N -X POST "$PROXY/v1/chat/completions" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"gpt-5-mini","messages":[{"role":"user","content":"Count from 1 to 5, digits only"}],"stream":true,"max_completion_tokens":32}'
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/messages" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"model":"gpt-5-mini","max_tokens":16,"messages":[{"role":"user","content":"Reply with the single word pong"}]}'
NEWKEY=$(curl -sS -X POST "$PROXY/key/generate" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"models":["typesafe/jev-1.13.0"],"key_alias":"lit9282c"}' | jq -r .key)
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/decisions" -H "Authorization: Bearer $NEWKEY" -H 'Content-Type: application/json' -d @body_jev.json
jq '.model="openrouter/typesafe/jev-1.13"' body_jev.json | curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/v1/decisions" -H "Authorization: Bearer $NEWKEY" -H 'Content-Type: application/json' -d @-
curl -sS -w '\nHTTP %{http_code}\n' -X POST "$PROXY/key/delete" -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"keys":["'"$NEWKEY"'"]}'
sleep 12; curl -sS -w '\nHTTP %{http_code}\n' "$PROXY/spend/logs" -H "Authorization: Bearer $KEY"

Before (`79f62db`, 2 workers, 127.0.0.1:58201)

== health_liveliness [base] HTTP 200 time=0.002277s
"I'm alive!"
== health_readiness [base] HTTP 200 time=0.008876s
{"status":"healthy","db":"connected"}
== health [base] HTTP 200 time=3.918201s
{"healthy_endpoints":[{"api_base":"http://127.0.0.1:17110","model":"typesafe/jev-1.13.0","model_id":"bdf8c33b93bd75589531f8913257b93fcf0d4d7016a702326afa85ec25ce6ccd"},{"api_base":"http://127.0.0.1:17111","model":"openrouter/typesafe/jev-1.13","model_id":"288d8517664dd66c8d7f5be20f6f1663bbe1613395f4dd982068b08cf0017096"},{"api_base":"http://127.0.0.1:17112","model":"perplexity/pplx-decider-v1-27b","model_id":"758f35280b2a2993103dc45555e0f643526f2ae30f62c083a6e580a0c00e9022"},{"api_base":"http://127.0.0.1:17113","model":"cloudflare/@cf/cloudflare/clef","model_id":"2bd843695cf0b9ff7a53a1037971f17f0ce7c4ed6280fa18734f1d574a2e588c"},{"model":"openai/gpt-5-mini","x-ratelimit-remaining-requests":"
{"healthy":["cloudflare/@cf/cloudflare/clef","openai/gpt-5-mini","openrouter/openai/gpt-5-mini","openrouter/typesafe/jev-1.13","perplexity/pplx-decider-v1-27b","typesafe/jev-1.13.0"],"unhealthy":[{"model":"cloudflare/@cf/cloudflare/clef","status":500,"error":"litellm.APIConnectionError: CloudflareException - Cannot connect to host 127.0.0.1:9 ssl:<ssl.SSLContext objec"},{"model":"openai/gpt-6-luna","status":400,"error":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openr"},{"model":"openrouter/openai/gpt-5-mini","status":500,"error":"litellm.APIError: APIError: OpenrouterException - Cannot connect to host 127.0.0.1:9 ssl:<ssl.SSLCont
== dec_typesafe [base] HTTP 200 time=0.291980s
{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.86},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"high":1.0,"low":0.0}},"confidence":{"type":"score","score":0.41,"confidence":0.18,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.59,"1":0.41}}},"usage":{"input_tokens":368,"output_tokens":62}}
== dec_openrouter [base] HTTP 200 time=0.382129s
{"model":"typesafe/jev-1.13-20260917","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"low":0.0,"high":1.0}},"confidence":{"type":"score","score":0.44,"confidence":0.13,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.56,"1":0.44}}},"usage":{"input_tokens":368,"output_tokens":62,"cost":0.000015456},"id":"gen-dec-1791426590-slOzGpkRqca3JXHXw2eP","provider":"TypeSafe"}
== dec_perplexity [base] HTTP 200 time=0.606359s
{"model":"pplx-decider-v1-27b","answers":{"defect":{"type":"noul","noul":0.9992161506933416},"severity":{"type":"choice","choice":"high","confidence":0.9962018103244494,"probabilities":{"low":0.001899094837775267,"high":0.9981009051622247}},"confidence":{"type":"score","score":0.6106911260350395,"confidence":0.2213822520700791,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.38930887396496044,"1":0.6106911260350395}}},"usage":{"input_tokens":299,"output_tokens":3}}
== dec_cloudflare [base] HTTP 200 time=0.434994s
{"model":"clef","answers":{"defect":{"type":"noul","noul":0.941},"severity":{"type":"choice","choice":"high","confidence":0.9366,"probabilities":{"low":0.0161,"high":0.9839}},"confidence":{"type":"score","score":0.9101,"confidence":0.6726,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.0899,"1":0.9101}}},"usage":{"input_tokens":287,"output_tokens":0}}
== dec_strands [base] HTTP 500 time=4.974885s
{"error":{"message":"litellm.APIConnectionError: Strands_deciderException - Server disconnected\n\nLiteLLM: model group 'strands_decider/strands-decider-2B-hobson-v19' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_typesafe_down [base] HTTP 500 time=4.008758s
{"error":{"message":"litellm.APIConnectionError: TypesafeException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7852c21aaa50> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'typesafe-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_perplexity_down [base] HTTP 500 time=5.059680s
{"error":{"message":"litellm.APIConnectionError: APIConnectionError: PerplexityException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x75bbf118af50> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'perplexity-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_openrouter_dec_down [base] HTTP 500 time=5.000062s
{"error":{"message":"litellm.APIConnectionError: APIConnectionError: OpenrouterException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7852c21aaa50> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'openrouter-dec-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_cloudflare_down [base] HTTP 500 time=4.520943s
{"error":{"message":"litellm.APIConnectionError: CloudflareException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7852c21aaa50> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'cloudflare-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_typesafe_badkey [base] HTTP 401 time=0.197754s
{"error":{"message":"litellm.AuthenticationError: TypesafeException - {\"detail\":{\"error_type\":\"authentication_error\",\"message\":\"Cannot authenticate with the server. Please check your API key and try again.\"}}\n\nLiteLLM: model group 'typesafe-badkey' failed with the error above. No fallback was attempted.","type":"authentication_error","param":null,"code":"401"}}
== dec_openai_luna [base] HTTP 400 time=0.047116s
{"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider\n\nLiteLLM: model group 'openai/gpt-6-luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
== dec_unknown_model [base] HTTP 400 time=0.046202s
{"error":{"message":"/decisions: Invalid model name passed in model=nosuch/jev-1.13.0. Call `/v1/models` to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provider_specific_fields":{"error":"/decisions: Invalid model name passed in model=nosuch/jev-1.13.0. Call `/v1/models` to view available models for your key."}}}
== dec_bad_body [base] HTTP 400 time=0.015823s
{"error":{"message":"litellm.BadRequestError: Invalid Decisions request: 1 validation error for DecisionsRequestBody\nstate\n  Field required [type=missing, input_value={'model': 'typesafe/jev-1...'instructions': 'ok?'}}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.13/v/missing","type":"invalid_request_error","param":null,"code":"400"}}
== chat_openrouter [base] HTTP 200 time=1.243266s
{"id":"gen-1791426615-hiVO0uP58TQZRUZtUUB0","created":1791426615,"model":"openrouter-chat","object":"chat.completion","choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","provider_specific_fields":{"reasoning":null,"reasoning_details":[{"type":"reasoning.encrypted","data":"gAAAAABqxwA4U0lZ9C5GYy8jvRKeUSlBQZlLZ1Pve9Xo0YEnEq7LmSaB_fS7OaWsj03hJQXO9Njv1q6-DUciao63_jg_fFIoyFwBORw7EjgO-6OogSiP5ohpDgluF6kMm17-bjDfu0gIZBxOM4tZG91oMaAUuHpxAqL8tWO5_WbTvCPPVJVNCHcRVqEL0GgkyAySTui4EnJKvgqGUZWs_a-F5biiZWpA3geRLzra5xshFBJ6gbTIaz4I6porgQqu8sgHV4yNW9mTfDIZoFSl3zKD8kQyu6dms-rZK-ytUq9Abr_igfUnZab7twWzmbHfyCbiEGk1dyVtT84ttgtyM2BbVoFc52QCPMGC_9KNByYNnq8x8YeH3Gvk3w_F6nS9Liv7fY1ywsxuxiCHk
== chat_openrouter_down [base] HTTP 500 time=5.187476s
{"error":{"message":"litellm.APIError: APIError: OpenrouterException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7852c21aaa50> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'openrouter-chat-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== chat_nonstream [base] HTTP 200 time=0.627066s
{"id":"chatcmpl-EWYBihILXBMAR3GoD7iUWCFlBhKFV","created":1791426622,"model":"gpt-5-mini","object":"chat.completion","choices":[{"finish_reason":"length","index":0,"message":{"content":"","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{}}],"usage":{"completion_tokens":16,"prompt_tokens":12,"total_tokens":28,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":16,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0}},"service_tier":"default"}
== chat_stream [base]
chunks=3 done=1
== messages [base] HTTP 200 time=0.898319s
{"id":"resp_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOm9wZW5haTttb2RlbF9pZDplMGFkYzFkODc1ZDA2MjAxMDUyMTNiMDBhYjU5ZjE1NDJjODQ5ZThlM2UxZmM0NDgwOWNjMDNmMzJmYjFjOTJjO3Jlc3BvbnNlX2lkOnJlc3BfMDkyNDIzMzc1MjhlZDY5MTAwNmFjNzAwNDA1ODM4ODdkMTg5MWI5YjExMGNjNzQxOTg=","type":"message","role":"assistant","model":"gpt-5-mini","stop_sequence":null,"usage":{"input_tokens":12,"output_tokens":0},"content":[{"type":"redacted_thinking","data":"litellm_encrypted_reasoning:gAAAAABqxwBAOp7cvGcKM1RoA6Kbv7XuBj7sz2EY1gU2c1E4bQiz8EXulhmq4noPR2w2pSNXpOOs-7G-n5QlsTMknWOX66g0GBQDk6yqzTn4G7bU6WBrEJNec5-lQTINsTbqmlcA5bJswuJ8VAtW2zFNQJpfEJEVslYD2C3E0IoRYaGL2bu8KLncILEQ5RwFEvVDwywYjtY6U1RSt5P7J9UKt42cuicbTiTRuH1XR65Nnfs7jdxIE3z10lN
== key_generate [base] HTTP 200 time=0.091705s
{"key_alias":"lit9282c-base","duration":null,"models":["typesafe/jev-1.13.0"],"spend":0.0,"max_budget":null,"user_id":null,"team_id":null,"agent_id":null,"max_parallel_requests":null,"metadata":{},"tpm_limit":null,"rpm_limit":null,"budget_duration":null,"budget_limits":null,"allowed_cache_controls":[],"config":{},"permissions":{},"model_max_budget":{},"budget_fallbacks":{},"model_rpm_limit":null,"model_tpm_limit":null,"mcp_rpm_limit":null,"tag_rpm_limit":null,"guardrails":null,"policies":null,"prompts":null,"blocked":null,"aliases":{},"object_permission":null,"key":"sk-<redacted>","tpd_limit":null,"default_estimated_output_tokens":null,"default_estimated_output_tokens_per_model":
== dec_with_generated_key [base] HTTP 200
{"model":"jev-1.13.0","answer_types":["noul","choice","score"],"usage":{"input_tokens":368,"output_tokens":62}}
== dec_generated_key_wrong_model [base] HTTP 403
"The requested model 'openrouter/typesafe/jev-1.13' is not available for this API key, or the model name is invalid. Check the models available to you and try ag"
== key_delete [base] HTTP 200 time=0.047656s
{"deleted_keys":["sk-<redacted>"]}
== spend_logs [base] HTTP 200 time=0.074394s
[{"request_id":"3e515287-cc88-4015-a6c3-bf3141b5a318","call_type":"adecisions","api_key":"8bb30ba28463fdf33ae62709cc832f68a21bcd86410b724b5562e8dcdfce4b61","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"startTime":"2026-10-08T02:30:25.474000Z","endTime":"2026-10-08T02:30:25.481000Z","request_duration_ms":7,"completionStartTime":"2026-10-08T02:30:25.480000Z","model":"openrouter/typesafe/jev-1.13","model_id":"288d8517664dd66c8d7f5be20f6f1663bbe1613395f4dd982068b08cf0017096","model_group":"openrouter/typesafe/jev-1.13","custom_llm_provider":"openrouter","api_base":"","user":"","metadata":{"status":"failure","user_agent":"curl/8.5.0","max_retries":null,"batch_models":null,
[{"model":"@cf/cloudflare/clef","call_type":"acompletion","spend":0.00003504,"total_tokens":146,"status":null,"api_base":"http://127.0.0.1:17113/ai/run/@cf/cloudflare/clef"},{"model":"cloudflare/@cf/cloudflare/clef","call_type":"adecisions","spend":0.00006887999999999999,"total_tokens":287,"status":null,"api_base":"http://127.0.0.1:17113/ai/run/@cf/cloudflare/clef"},{"model":"cloudflare/@cf/cloudflare/clef","call_type":"adecisions","spend":0.0,"total_tokens":0,"status":"failure","api_base":"http://127.0.0.1:9/ai/run/@cf/cloudflare/clef"},{"model":"jev-1.13.0","call_type":"acompletion","spend":0.000011592,"total_tokens":296,"status":null,"api_base":"http://127.0.0.1:17110/v1/systemone"},{"mod

After (`bd81e17`, 2 workers, 127.0.0.1:52979)

== health_liveliness [head] HTTP 200 time=0.002615s
"I'm alive!"
== health_readiness [head] HTTP 200 time=0.032777s
{"status":"healthy","db":"connected"}
== health [head] HTTP 200 time=3.721108s
{"healthy_endpoints":[{"api_base":"http://127.0.0.1:17120","model":"typesafe/jev-1.13.0","model_id":"b3bf66bb570cdeb0841046c68b340ef970b74a51da1a3c4c8cb5e510bf22573b"},{"api_base":"http://127.0.0.1:17121","model":"openrouter/typesafe/jev-1.13","model_id":"9e77a85ac3a7ae9602f7fe0cfd2a81cc6aa88355a5f30c740f1e22850f2cf9de"},{"api_base":"http://127.0.0.1:17122","model":"perplexity/pplx-decider-v1-27b","model_id":"f3f0fcf148bf082071f0f6a88566bee2c7e9c831493482b7eb7301b48b760195"},{"api_base":"http://127.0.0.1:17123","model":"cloudflare/@cf/cloudflare/clef","model_id":"11ff5b8958c134b55e85e30a6507a2d46f96c09bd4dc2fddd17c0a16b45c88e8"},{"model":"openai/gpt-5-mini","x-ratelimit-remaining-requests":"
{"healthy":["cloudflare/@cf/cloudflare/clef","openai/gpt-5-mini","openrouter/openai/gpt-5-mini","openrouter/typesafe/jev-1.13","perplexity/pplx-decider-v1-27b","typesafe/jev-1.13.0"],"unhealthy":[{"model":"cloudflare/@cf/cloudflare/clef","status":500,"error":"litellm.APIConnectionError: CloudflareException - Cannot connect to host 127.0.0.1:9 ssl:<ssl.SSLContext objec"},{"model":"openai/gpt-6-luna","status":400,"error":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: openrouter, perplexity, typ"},{"model":"openrouter/openai/gpt-5-mini","status":500,"error":"litellm.APIConnectionError: APIConnectionError: OpenrouterException - Cannot connect to host 127.0.0.
== dec_typesafe [head] HTTP 200 time=0.334929s
{"model":"jev-1.13.0","answers":{"defect":{"type":"noul","noul":0.87},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"high":1.0,"low":0.0}},"confidence":{"type":"score","score":0.43,"confidence":0.14,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.57,"1":0.43}}},"usage":{"input_tokens":368,"output_tokens":62}}
== dec_openrouter [head] HTTP 200 time=0.541905s
{"model":"typesafe/jev-1.13-20260917","answers":{"defect":{"type":"noul","noul":0.86},"severity":{"type":"choice","choice":"high","confidence":1.0,"probabilities":{"high":1.0,"low":0.0}},"confidence":{"type":"score","score":0.46,"confidence":0.08,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.54,"1":0.46}}},"usage":{"input_tokens":368,"output_tokens":62,"cost":0.000015456},"id":"gen-dec-1791426642-RvbTZ3LqnTvtcPFSLj5w","provider":"TypeSafe"}
== dec_perplexity [head] HTTP 200 time=0.469211s
{"model":"pplx-decider-v1-27b","answers":{"defect":{"type":"noul","noul":0.9991146448181556},"severity":{"type":"choice","choice":"high","confidence":0.9960951748622859,"probabilities":{"low":0.001952412568856984,"high":0.998047587431143}},"confidence":{"type":"score","score":0.6106911260350395,"confidence":0.2213822520700791,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.38930887396496044,"1":0.6106911260350395}}},"usage":{"input_tokens":299,"output_tokens":3}}
== dec_cloudflare [head] HTTP 200 time=0.522265s
{"model":"clef","answers":{"defect":{"type":"noul","noul":0.941},"severity":{"type":"choice","choice":"high","confidence":0.9366,"probabilities":{"low":0.0161,"high":0.9839}},"confidence":{"type":"score","score":0.9101,"confidence":0.6726,"legend":{"0":"Not sure","1":"Certain"},"probabilities":{"0":0.0899,"1":0.9101}}},"usage":{"input_tokens":287,"output_tokens":0}}
== dec_strands [head] HTTP 500 time=4.938977s
{"error":{"message":"litellm.APIConnectionError: Strands_deciderException - Server disconnected\n\nLiteLLM: model group 'strands_decider/strands-decider-2B-hobson-v19' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_typesafe_down [head] HTTP 500 time=3.933936s
{"error":{"message":"litellm.APIConnectionError: TypesafeException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7031bbbde8d0> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'typesafe-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_perplexity_down [head] HTTP 500 time=5.221196s
{"error":{"message":"litellm.APIConnectionError: PerplexityException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7031bbbde8d0> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'perplexity-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_openrouter_dec_down [head] HTTP 500 time=4.686744s
{"error":{"message":"litellm.APIConnectionError: OpenrouterException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x74edcef9e8d0> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'openrouter-dec-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_cloudflare_down [head] HTTP 500 time=4.946371s
{"error":{"message":"litellm.APIConnectionError: CloudflareException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x74edcef9e8d0> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'cloudflare-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== dec_typesafe_badkey [head] HTTP 401 time=0.193435s
{"error":{"message":"litellm.AuthenticationError: TypesafeException - {\"detail\":{\"error_type\":\"authentication_error\",\"message\":\"Cannot authenticate with the server. Please check your API key and try again.\"}}\n\nLiteLLM: model group 'typesafe-badkey' failed with the error above. No fallback was attempted.","type":"authentication_error","param":null,"code":"401"}}
== dec_openai_luna [head] HTTP 400 time=0.031572s
{"error":{"message":"litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: openrouter, perplexity, typesafe, strands_decider, cloudflare\n\nLiteLLM: model group 'openai/gpt-6-luna' failed with the error above. No fallback was attempted.","type":"invalid_request_error","param":null,"code":"400"}}
== dec_unknown_model [head] HTTP 400 time=0.043976s
{"error":{"message":"/decisions: Invalid model name passed in model=nosuch/jev-1.13.0. Call `/v1/models` to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provider_specific_fields":{"error":"/decisions: Invalid model name passed in model=nosuch/jev-1.13.0. Call `/v1/models` to view available models for your key."}}}
== dec_bad_body [head] HTTP 400 time=0.019479s
{"error":{"message":"litellm.BadRequestError: Invalid Decisions request: 1 validation error for DecisionsRequestBody\nstate\n  Field required [type=missing, input_value={'model': 'typesafe/jev-1...'instructions': 'ok?'}}}, input_type=dict]\n    For further information visit https://errors.pydantic.dev/2.13/v/missing","type":"invalid_request_error","param":null,"code":"400"}}
== chat_openrouter [head] HTTP 200 time=1.272814s
{"id":"gen-1791426667-KLz8PQtibklwf6OuWY9v","created":1791426667,"model":"openrouter-chat","object":"chat.completion","choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","provider_specific_fields":{"refusal":null,"reasoning_details":[{"type":"reasoning.encrypted","data":"gAAAAABqxwBsuKFVFiINh85Gd9x3yi-u-pZNpldd-i47M8n4a3shhYttFlrZtwsUZlKLMc0CPZlvptbw-rjqABP0xvLfxO8_tMo5Dv_qbJn04lDZRfCV1odwA_IeJgwIFR77FtD3b7geMsAmUYaa_Kb4cxjFp-Ht_hOevExegBTFJhtiChabELAp4II3c9gO0BrKfcaocPpFwLsE-CeFqwVT5CkCCPUqrrfqSaIOJPirPEq4cjEcjcVY7M9d9uoIASTw5rOPv1iJYg4_Ouy4VBZu66U2c3LVFTpEQ3miIjUp2ZGR8Z63yPOJrXfRbKdvkoGYvXYO_g3xLM8_c2WWZCDDI-ne5pikUjGLeNIgdmvf3_LNeCBWKmvfbPbR8LEaMDok-7HWk1zXr8g3QXI
== chat_openrouter_down [head] HTTP 500 time=3.998821s
{"error":{"message":"litellm.APIConnectionError: APIConnectionError: OpenrouterException - Cannot connect to host REDACTED:9 ssl:<ssl.SSLContext object at 0x7031bbbde8d0> [Connect call failed ('REDACTED', 9)]\n\nLiteLLM: model group 'openrouter-chat-down' failed with the error above. No fallback was attempted.","type":"internal_server_error","param":null,"code":"500"}}
== chat_nonstream [head] HTTP 200 time=0.838905s
{"id":"chatcmpl-EWYCXX80xKd4IPMLKzY5DSJGCx9mU","created":1791426673,"model":"gpt-5-mini","object":"chat.completion","choices":[{"finish_reason":"length","index":0,"message":{"content":"","role":"assistant","provider_specific_fields":{"refusal":null},"annotations":[]},"provider_specific_fields":{}}],"usage":{"completion_tokens":16,"prompt_tokens":12,"total_tokens":28,"completion_tokens_details":{"accepted_prediction_tokens":0,"audio_tokens":0,"reasoning_tokens":16,"rejected_prediction_tokens":0},"prompt_tokens_details":{"audio_tokens":0,"cached_tokens":0}},"service_tier":"default"}
== chat_stream [head]
chunks=3 done=1
== messages [head] HTTP 200 time=1.478980s
{"id":"resp_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOm9wZW5haTttb2RlbF9pZDplMGFkYzFkODc1ZDA2MjAxMDUyMTNiMDBhYjU5ZjE1NDJjODQ5ZThlM2UxZmM0NDgwOWNjMDNmMzJmYjFjOTJjO3Jlc3BvbnNlX2lkOnJlc3BfMGI2MjgyOTdhMjk0MDVhYTAwNmFjNzAwNzJhYmY4ODdkMTllMzQ3NmNkNmE4NzEwODI=","type":"message","role":"assistant","model":"gpt-5-mini","stop_sequence":null,"usage":{"input_tokens":12,"output_tokens":0},"content":[{"type":"redacted_thinking","data":"litellm_encrypted_reasoning:gAAAAABqxwBzxaKd6o-vOk2fPoNN7DDTiKg2OJPzYLhyVIFvRB3nJSLHC8TV84pqH9N4WGrzootKmRNUFVPlWWVqSrTG1qThKb7NTRKBMfRG21CUZ6YP4_gJXYB0akJc3Crtt64K3cJ56H-crCye5AnrnoWNbfVKya_78I3DPe5obnmSWOkISfvdYSxRDPcaXbWPFOoPH2rTZroLGU9PHD4ABgtN9XdWmy6s1i5Wahnk40MFE4tTQVXeEkf
== key_generate [head] HTTP 200 time=0.094414s
{"key_alias":"lit9282c-head","duration":null,"models":["typesafe/jev-1.13.0"],"spend":0.0,"max_budget":null,"user_id":null,"team_id":null,"agent_id":null,"max_parallel_requests":null,"metadata":{},"tpm_limit":null,"rpm_limit":null,"budget_duration":null,"budget_limits":null,"allowed_cache_controls":[],"config":{},"permissions":{},"model_max_budget":{},"budget_fallbacks":{},"model_rpm_limit":null,"model_tpm_limit":null,"mcp_rpm_limit":null,"tag_rpm_limit":null,"guardrails":null,"policies":null,"prompts":null,"blocked":null,"aliases":{},"object_permission":null,"key":"sk-<redacted>","tpd_limit":null,"default_estimated_output_tokens":null,"default_estimated_output_tokens_per_model":
== dec_with_generated_key [head] HTTP 200
{"model":"jev-1.13.0","answer_types":["noul","choice","score"],"usage":{"input_tokens":368,"output_tokens":62}}
== dec_generated_key_wrong_model [head] HTTP 403
"The requested model 'openrouter/typesafe/jev-1.13' is not available for this API key, or the model name is invalid. Check the models available to you and try ag"
== key_delete [head] HTTP 200 time=0.039793s
{"deleted_keys":["sk-<redacted>"]}
== spend_logs [head] HTTP 200 time=0.073326s
[{"request_id":"e2094e03-d3d9-499e-a5fb-194f76e8feae","call_type":"adecisions","api_key":"8ed6082c3a8d72e781a1f62a504f18566bfee1dcde2843c43232bd7a0b0ea73a","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"startTime":"2026-10-08T02:31:16.439000Z","endTime":"2026-10-08T02:31:16.443000Z","request_duration_ms":4,"completionStartTime":"2026-10-08T02:31:16.442000Z","model":"openrouter/typesafe/jev-1.13","model_id":"9e77a85ac3a7ae9602f7fe0cfd2a81cc6aa88355a5f30c740f1e22850f2cf9de","model_group":"openrouter/typesafe/jev-1.13","custom_llm_provider":"openrouter","api_base":"","user":"","metadata":{"status":"failure","user_agent":"curl/8.5.0","max_retries":null,"batch_models":null,
[{"model":"@cf/cloudflare/clef","call_type":"acompletion","spend":0.00003504,"total_tokens":146,"status":null,"api_base":"http://127.0.0.1:17123/ai/run/@cf/cloudflare/clef"},{"model":"cloudflare/@cf/cloudflare/clef","call_type":"adecisions","spend":0.00006887999999999999,"total_tokens":287,"status":null,"api_base":"http://127.0.0.1:17123/ai/run/@cf/cloudflare/clef"},{"model":"cloudflare/@cf/cloudflare/clef","call_type":"adecisions","spend":0.0,"total_tokens":0,"status":"failure","api_base":"http://127.0.0.1:9/ai/run/@cf/cloudflare/clef"},{"model":"jev-1.13.0","call_type":"acompletion","spend":0.000011592,"total_tokens":296,"status":null,"api_base":"http://127.0.0.1:17120/v1/systemone"},{"mod

Outbound wire

Recorder logs normalized with jq -Sc 'del(.ts, .upstream_ms)' (the recorder redacts authorization). Before logged typesafe 6, openrouter 2, perplexity 2, cloudflare 2, strands_decider 4 requests and After typesafe 6, openrouter 2, perplexity 2, cloudflare 2, strands_decider 4 (the /health probes, the dec_* cells and the generated-key cells; the down deployments point at a closed port, so nothing reaches a recorder for them) and the normalized diff is empty for all five. One record per provider from the After leg, the dec_* body:

{"body":"{\"model\":\"jev-1.13.0\",\"state\":\"Ticket (billing): The export job hangs at 99% and never finishes\",\"questions\":{\"defect\":{\"type\":\"noul\",\"instructions\":\"Is this a defect?\"},\"severity\":{\"type\":\"choice\",\"instructions\":\"How severe is it?\",\"criteria\":{\"low\":\"Cosmetic\",\"high\":\"Blocks the user\"}},\"confidence\":{\"type\":\"score\",\"instructions\":\"How confident are you?\",\"criteria\":[\"Not sure\",\"Certain\"]}}}","headers":{"accept":"*/*","authorization":"<redacted len=115>","content-type":"application/json","user-agent":"litellm/1.106.0"},"method":"POST","path":"/v1/systemone","status":200}
{"body":"{\"model\":\"typesafe/jev-1.13\",\"state\":\"Ticket (billing): The export job hangs at 99% and never finishes\",\"questions\":{\"defect\":{\"type\":\"noul\",\"instructions\":\"Is this a defect?\"},\"severity\":{\"type\":\"choice\",\"instructions\":\"How severe is it?\",\"criteria\":{\"low\":\"Cosmetic\",\"high\":\"Blocks the user\"}},\"confidence\":{\"type\":\"score\",\"instructions\":\"How confident are you?\",\"criteria\":[\"Not sure\",\"Certain\"]}}}","headers":{"accept":"*/*","authorization":"<redacted len=80>","content-type":"application/json","user-agent":"litellm/1.106.0"},"method":"POST","path":"/alpha/decisions","status":200}
{"body":"{\"model\":\"pplx-decider-v1-27b\",\"state\":\"Ticket (billing): The export job hangs at 99% and never finishes\",\"questions\":{\"defect\":{\"type\":\"noul\",\"instructions\":\"Is this a defect?\"},\"severity\":{\"type\":\"choice\",\"instructions\":\"How severe is it?\",\"criteria\":{\"low\":\"Cosmetic\",\"high\":\"Blocks the user\"}},\"confidence\":{\"type\":\"score\",\"instructions\":\"How confident are you?\",\"criteria\":[\"Not sure\",\"Certain\"]}}}","headers":{"accept":"*/*","authorization":"<redacted len=60>","content-type":"application/json","user-agent":"litellm/1.106.0"},"method":"POST","path":"/v1/decisions","status":200}
{"body":"{\"model\":\"clef\",\"state\":\"Ticket (billing): The export job hangs at 99% and never finishes\",\"questions\":{\"defect\":{\"type\":\"noul\",\"instructions\":\"Is this a defect?\"},\"severity\":{\"type\":\"choice\",\"instructions\":\"How severe is it?\",\"criteria\":{\"low\":\"Cosmetic\",\"high\":\"Blocks the user\"}},\"confidence\":{\"type\":\"score\",\"instructions\":\"How confident are you?\",\"criteria\":[\"Not sure\",\"Certain\"]}}}","headers":{"accept":"*/*","authorization":"<redacted len=60>","content-type":"application/json","user-agent":"litellm/1.106.0"},"method":"POST","path":"/ai/run/@cf/cloudflare/clef","status":200}
{"body":"{\"model\":\"strands-decider-2B-hobson-v19\",\"state\":\"Ticket (billing): The export job hangs at 99% and never finishes\",\"questions\":{\"defect\":{\"type\":\"noul\",\"instructions\":\"Is this a defect?\"},\"severity\":{\"type\":\"choice\",\"instructions\":\"How severe is it?\",\"criteria\":{\"low\":\"Cosmetic\",\"high\":\"Blocks the user\"}},\"confidence\":{\"type\":\"score\",\"instructions\":\"How confident are you?\",\"criteria\":[\"Not sure\",\"Certain\"]}}}","headers":{"accept":"*/*","content-type":"application/json","user-agent":"litellm/1.106.0"},"method":"POST","path":"/v1/systemone","status":null}

Spend rows

/spend/logs rows for the decisions deployments and the OpenRouter chat deployments, recorder ports replaced by PORT; the Before and After files are byte for byte identical:

[{"model":"@cf/cloudflare/clef","status":null,"spend":0.00003504,"total_tokens":146,"prompt_tokens":146,"completion_tokens":0,"api_base":"http://127.0.0.1:PORT/ai/run/@cf/cloudflare/clef","provider":"cloudflare"},{"model":"cloudflare/@cf/cloudflare/clef","status":null,"spend":0.00006887999999999999,"total_tokens":287,"prompt_tokens":287,"completion_tokens":0,"api_base":"http://127.0.0.1:PORT/ai/run/@cf/cloudflare/clef","provider":"cloudflare"},{"model":"cloudflare/@cf/cloudflare/clef","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:9/ai/run/@cf/cloudflare/clef","provider":"cloudflare"},{"model":"jev-1.13.0","status":null,"spend":0.000011592,"total_tokens":296,"prompt_tokens":276,"completion_tokens":20,"api_base":"http://127.0.0.1:PORT/v1/systemone","provider":"typesafe"},{"model":"openai/gpt-6-luna","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"","provider":""},{"model":"openrouter/openai/gpt-5-mini","status":null,"spend":0.000003,"total_tokens":12,"prompt_tokens":12,"completion_tokens":0,"api_base":"https://openrouter.ai/api/v1/chat/completions","provider":"openrouter"},{"model":"openrouter/openai/gpt-5-mini","status":"failure","spend":0.0,"total_tokens":13,"prompt_tokens":13,"completion_tokens":0,"api_base":"http://127.0.0.1:9/chat/completions","provider":"openrouter"},{"model":"openrouter/typesafe/jev-1.13","status":null,"spend":0.000015456,"total_tokens":430,"prompt_tokens":368,"completion_tokens":62,"api_base":"http://127.0.0.1:PORT/alpha/decisions","provider":"openrouter"},{"model":"openrouter/typesafe/jev-1.13","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"","provider":"openrouter"},{"model":"openrouter/typesafe/jev-1.13","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:9/alpha/decisions","provider":"openrouter"},{"model":"perplexity/pplx-decider-v1-27b","status":null,"spend":0.00001196,"total_tokens":302,"prompt_tokens":299,"completion_tokens":3,"api_base":"http://127.0.0.1:PORT/v1/decisions","provider":"perplexity"},{"model":"perplexity/pplx-decider-v1-27b","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:9/v1/decisions","provider":"perplexity"},{"model":"pplx-decider-v1-27b","status":null,"spend":0.00000356,"total_tokens":90,"prompt_tokens":89,"completion_tokens":1,"api_base":"http://127.0.0.1:PORT/v1/decisions","provider":"perplexity"},{"model":"strands_decider/strands-decider-2B-hobson-v19","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:PORT/v1/systemone","provider":"strands_decider"},{"model":"typesafe/jev-1.13","status":null,"spend":0.000011592,"total_tokens":296,"prompt_tokens":276,"completion_tokens":20,"api_base":"http://127.0.0.1:PORT/alpha/decisions","provider":"openrouter"},{"model":"typesafe/jev-1.13.0","status":null,"spend":0.000015456,"total_tokens":430,"prompt_tokens":368,"completion_tokens":62,"api_base":"http://127.0.0.1:PORT/v1/systemone","provider":"typesafe"},{"model":"typesafe/jev-1.13.0","status":null,"spend":0.000015456,"total_tokens":430,"prompt_tokens":368,"completion_tokens":62,"api_base":"http://127.0.0.1:PORT/v1/systemone","provider":"typesafe"},{"model":"typesafe/jev-1.13.0","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"","provider":""},{"model":"typesafe/jev-1.13.0","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:9/v1/systemone","provider":"typesafe"},{"model":"typesafe/jev-1.13.0","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"http://127.0.0.1:PORT/v1/systemone","provider":"typesafe"},{"model":"unknown-model","status":"failure","spend":0.0,"total_tokens":0,"prompt_tokens":0,"completion_tokens":0,"api_base":"","provider":""}]

SDK client

litellm.decisions from each leg's own checkout against the same recorders and a closed port (ab3/sdk_cell.py; the TypeSafe key is read from the box's .env, never printed):

sdk_sync_typesafe /home/mateo/rigs/lit9282c/base/litellm/__init__.py jev-1.13.0 ['defect'] 287 21 typesafe
sdk_sync_typesafe_down APIConnectionError 500 litellm.APIConnectionError: TypesafeException - [Errno 111] Connection refused
sdk_sync_perplexity_down APIConnectionError 500 litellm.APIConnectionError: APIConnectionError: PerplexityException - [Errno 111] Connection refused
sdk_sync_openai_luna BadRequestError 400 litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: perplexity, typesafe, openrouter, cloudflare, strands_decider

sdk_sync_typesafe /home/mateo/rigs/lit9282c/head/litellm/__init__.py jev-1.13.0 ['defect'] 287 21 typesafe
sdk_sync_typesafe_down APIConnectionError 500 litellm.APIConnectionError: TypesafeException - [Errno 111] Connection refused
sdk_sync_perplexity_down APIConnectionError 500 litellm.APIConnectionError: PerplexityException - [Errno 111] Connection refused
sdk_sync_openai_luna BadRequestError 400 litellm.BadRequestError: Unknown Decisions provider 'openai'. Supported providers: openrouter, perplexity, typesafe, strands_decider, cloudflare

Observations

Jev answers match on both legs except the sampled score on TypeSafe and OpenRouter; Perplexity and Cloudflare answers are identical to the digit
Unreachable TypeSafe, Perplexity, OpenRouter and Cloudflare decisions deployments: 500 litellm.APIConnectionError on both legs, through curl and the SDK; at 5276719, before the audit fix in the TLDR, the Perplexity one said litellm.InternalServerError. Before, the Perplexity and OpenRouter messages carried the class name twice (litellm.APIConnectionError: APIConnectionError: PerplexityException - ...) where TypeSafe and Cloudflare carried it once; After, all four read litellm.APIConnectionError: <Provider>Exception - ...
Bad TypeSafe key: 401 litellm.AuthenticationError on both legs
openai/gpt-6-luna: 400 Unknown Decisions provider 'openai' on both legs, the supported list in a different order
Unknown model and body without state: the same 400s on both legs
OpenRouter chat: 200 on both legs; OpenRouter chat at a down api_base: 500 on both legs, litellm.APIError: APIError: OpenrouterException Before and litellm.APIConnectionError: APIConnectionError: OpenrouterException After (the review fix in the TLDR)
/health: the same healthy set on both legs; unhealthy openai/gpt-6-luna 400, the four down decisions deployments and the down OpenRouter chat deployment 500 (error classes as above) and the StrandsDecider 500 (no service behind its recorder)
gpt-5-mini chat, streaming chat and /v1/messages unchanged
Generated key scoped to typesafe/jev-1.13.0: decisions 200 with its model and 403 for another model, on both legs
Spend rows identical, adecisions with the provider path in api_base

Verdict: PASS (Before = After at every cell except the three intended differences named above: the error class of an unreachable OpenRouter chat deployment, the order of the supported-provider list, and the duplicated class name dropped from the Perplexity and OpenRouter decisions connection errors)

/audit

Merge base 79f62db, head bd81e17, rig lit9282aud on the GCE box (three legs, two workers each, own Postgres, Redis and scripted upstream per leg), lanes base, head-run1, head-run2, ctl-providers, ctl-management and ctl-cost; summaries round-lanes.json

Row	Cell	base	head run1	head run2	ctl	Expected (base / head)	Result
A3	`providers/test_decisions_wire.py::test_sdk_sync_and_async_clients_send_the_same_request`	green	green	green	-	green / green	PASS
B3	`providers/test_decisions_wire.py::test_unreachable_upstream_fails_only_its_own_deployment[perplexity]`	green	green	green	-	green / green	PASS
B3	`providers/test_decisions_wire.py::test_unreachable_upstream_fails_only_its_own_deployment[typesafe]`	green	green	green	-	green / green	PASS
B3	`providers/test_decisions_wire.py::test_unreachable_upstream_fails_only_its_own_deployment[openrouter]`	green	green	green	-	green / green	PASS
B3	`providers/test_decisions_wire.py::test_unreachable_upstream_fails_only_its_own_deployment[strands_decider]`	green	green	green	-	green / green	PASS
B3	`providers/test_decisions_wire.py::test_unreachable_upstream_fails_only_its_own_deployment[cloudflare]`	green	green	green	-	green / green	PASS
B1	`providers/test_decisions_wire.py::test_an_unreachable_openrouter_deployment_reports_a_connection_error_on_chat_embeddings_and_decisions`	red	green	green	-	red / green	PASS
B2	`providers/test_decisions_wire.py::test_sdk_openrouter_connection_failures_raise_a_connection_error`	red	green	green	-	red / green	PASS
B4	`providers/test_decisions_wire.py::test_a_deployment_whose_provider_has_no_decisions_support_is_refused_naming_every_supported_provider`	green	green	green	-	green / green	PASS
A2	`translation/decisions/basic/test_decisions_basic_cloudflare.py::test_decisions_basic_cloudflare[cloudflare/@cf/cloudflare/clef-basic]`	green	green	green	-	green / green	PASS
A2	`translation/decisions/basic/test_decisions_basic_openrouter.py::test_decisions_basic_openrouter[openrouter/typesafe/jev-1.13-basic]`	green	green	green	-	green / green	PASS
A2	`translation/decisions/basic/test_decisions_basic_perplexity.py::test_decisions_basic_perplexity[perplexity/pplx-decider-v1-27b-basic]`	green	green	green	-	green / green	PASS
A2	`translation/decisions/basic/test_decisions_basic_strands_decider.py::test_decisions_basic_strands_decider[strands_decider/strands-decider-2B-hobson-v19-basic]`	green	green	green	-	green / green	PASS
A2	`translation/decisions/basic/test_decisions_basic_typesafe.py::test_decisions_basic_typesafe[typesafe/jev-1.13.0-basic]`	green	green	green	-	green / green	PASS
A1	`providers/test_decisions_wire.py::test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map[perplexity]`	-	-	-	green	green / green	PASS
A1	`providers/test_decisions_wire.py::test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map[typesafe]`	-	-	-	green	green / green	PASS
A1	`providers/test_decisions_wire.py::test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map[openrouter]`	-	-	-	green	green / green	PASS
A1	`providers/test_decisions_wire.py::test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map[strands_decider]`	-	-	-	green	green / green	PASS
A1	`providers/test_decisions_wire.py::test_each_provider_gets_its_own_path_key_and_body_and_is_billed_from_the_cost_map[cloudflare]`	-	-	-	green	green / green	PASS
D4	`providers/test_decisions_wire.py::test_repeated_identical_requests_each_reach_the_upstream_and_are_each_billed`	-	-	-	green	green / green	PASS
D4	`providers/test_decisions_wire.py::test_gateway_only_fields_stay_at_the_gateway_and_tags_reach_the_spend_log`	-	-	-	green	green / green	PASS
C3	`providers/test_decisions_wire.py::test_invalid_bodies_are_refused_at_the_gateway_without_an_upstream_call`	-	-	-	green	green / green	PASS
C4	`providers/test_decisions_wire.py::test_unknown_model_is_refused_like_chat`	-	-	-	green	green / green	PASS
C4	`providers/test_decisions_wire.py::test_key_checks_match_chat`	-	-	-	green	green / green	PASS
C4	`providers/test_decisions_wire.py::test_request_body_api_base_is_refused_like_chat_without_an_upstream_call`	-	-	-	green	green / green	PASS
D1	`providers/test_decisions_wire.py::test_a_deployment_without_a_key_sends_the_provider_env_key_to_its_configured_api_base`	-	-	-	green	green / green	PASS
D2	`providers/test_decisions_wire.py::test_a_deployment_opted_into_client_api_base_sends_decisions_and_chat_to_the_body_api_base`	-	-	-	green	green / green	PASS
D3	`providers/test_decisions_wire.py::test_a_config_pass_through_at_v1_decisions_keeps_answering_and_the_native_api_serves_decisions`	-	-	-	green	green / green	PASS
C2	`providers/test_decisions_wire.py::test_upstream_errors_keep_their_status_and_log_an_unbilled_failure[401]`	-	-	-	green	green / green	PASS
C2	`providers/test_decisions_wire.py::test_upstream_errors_keep_their_status_and_log_an_unbilled_failure[429]`	-	-	-	green	green / green	PASS
C2	`providers/test_decisions_wire.py::test_upstream_errors_keep_their_status_and_log_an_unbilled_failure[500]`	-	-	-	green	green / green	PASS
C1	`providers/test_decisions_wire.py::test_upstream_success_without_answers_is_a_gateway_side_server_error`	-	-	-	green	green / green	PASS
E1	`providers/test_decisions_chaos.py::test_burst_over_both_routes_bills_each_call_once_with_its_own_status`	-	-	-	green	green / green	PASS
E1	`providers/test_decisions_chaos.py::test_worker_sigkill_mid_burst_leaves_the_sibling_serving_the_default_model`	-	-	-	green	green / green	PASS
E1	`providers/test_decisions_chaos.py::test_upstream_outage_fails_its_calls_and_recovery_on_the_same_port_restores_them`	-	-	-	green	green / green	PASS
F1	`management/test_model_health_check.py::test_evaluation_mode_health_check_resolves_the_mode_from_the_cost_map_and_sends_the_default_probe`	-	-	-	green	green / green	PASS
F1	`management/test_model_health_check.py::test_evaluation_mode_health_check_sends_the_configured_state_and_questions`	-	-	-	green	green / green	PASS
F1	`management/test_model_health_check.py::test_evaluation_mode_health_check_of_the_self_hosted_strands_model_resolves_the_mode_from_the_cost_map`	-	-	-	green	green / green	PASS
F2	`cost_calculation/test_cost_tracking.py::test_case_bills_expected_cost[perplexity/pplx-decider-v1-27b-decisions]`	-	-	-	green	green / green	PASS

| L1 | live: TypeSafe, OpenRouter, Perplexity, Cloudflare reached with real keys through the Before and After proxies above (/live-pr-risk A/B at bd81e17 vs 79f62db, outbound wire diff empty, replies equal but for the sampled score, spend rows identical) | PASS |
| L2 | live: StrandsDecider has no public service; its outbound request (POST /v1/systemone, no Authorization header, Jev body) is in the After leg's recorder log and its reply shape is pinned by the A1 and A2 cells | PASS |

Every cell ran at bd81e17; the two head runs collected and passed the same selections with no skips and no retries

Last updated: bd81e17 (the Before/After legs, the SDK cell and the audit all ran at this tip)

bd81e17 passes /audit

Type

🧹 Refactoring

Caveats (if any)

Low

An OpenRouter chat, embeddings or decisions deployment that cannot be reached now answers 500 litellm.APIConnectionError instead of 500 litellm.APIError; the status is unchanged and the new class is the one every other provider already reports, chosen in the review fix 7933ace named in the TLDR
The Unknown Decisions provider 400 lists the five supported providers in registration order (openrouter, perplexity, typesafe, strands_decider, cloudflare) instead of the old dict order; status and set are unchanged
The 500 for an unreachable Perplexity or OpenRouter decisions deployment reads litellm.APIConnectionError: PerplexityException - ... instead of litellm.APIConnectionError: APIConnectionError: PerplexityException - ..., the class name once like TypeSafe and Cloudflare already had; class and status are unchanged
The score answer's score and probabilities differ between Before and After by a few hundredths because the model is sampled, not because of this PR; noul, choice, legend and usage match
StrandsDecider is a self-hosted model with no public service, so its live rows prove the outbound request (path, no Authorization header, body) and not a reply; the scripted-upstream cells cover its reply shape
The complexity router and the TypeSafe guardrail still post their own Jev bodies instead of calling litellm.adecisions; that migration is tracked separately on LIT-9282
hosted_vllm, laya and bespoke are not registered yet; a request for them gets the same Unknown Decisions provider 400 as before

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/a7a7d712811643b1a6416531d7d99cfa
Open in Devin Desktop: https://app.devin.ai/desktop/session/a7a7d712811643b1a6416531d7d99cfa?variant=devin
Requested by: @kerry-berri

Note

Medium Risk
Touches request routing, API key resolution, and response parsing for /v1/decisions; behavior is intended to be unchanged but errors and provider resolution paths shifted to shared infrastructure.

Overview
Refactors /v1/decisions so it no longer uses a hardcoded DECISIONS_ENDPOINTS map and inline httpx in litellm/decisions/main.py. Calls now go through get_llm_provider, ProviderConfigManager.get_provider_decisions_config, and BaseLLMHTTPHandler.decisions / adecisions, matching other LiteLLM provider APIs.

Each supported provider (perplexity, typesafe, openrouter, cloudflare, strands_decider) gets a BaseDecisionsConfig subclass (replacing JevCompatibleDecisionsEndpoint singletons) for URL, auth/env resolution, request body, and response unwrapping. The base class centralizes default Jev/System One wire behavior; Cloudflare keeps custom URL and result envelope handling.

DECISIONS_ENDPOINTS is removed from the public module exports. DecisionsResponse.set_hidden_params is added for metadata after transforms. Malformed upstream payloads without valid answers now surface as InternalServerError during response validation.

Tests: per-provider TranslationTestCase bases and integration tests pin expected paths/headers/bodies; unit tests cover provider rejection, empty custom_llm_provider falling back to model prefix, Router deployment API keys, and bad upstream responses. Proxy integration config adds mock entries for all five providers.

^{Reviewed by Cursor Bugbot for commit ee4dd89. Bugbot is set up for automated code reviews on this repo. Configure here.}

bd81e17 passes /live-pr-risk

devin-ai-integration · 2026-10-07T18:49:34Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

litellm-stampbot · 2026-10-07T18:49:58Z

@greptileai

litellm-stampbot · 2026-10-07T18:49:59Z

bugbot run

CLAassistant · 2026-10-07T18:50:00Z

All committers have signed the CLA.

cursor

Stale Bugbot comment from a previous run.

greptile-apps · 2026-10-07T18:59:50Z

Confidence Score: 5/5

[High risk] Refactors decisions endpoint routing through shared HTTP handler.

The PR appears safe to merge, with the previous numbered finding fixed and no new actionable issues

What we checked:

Timeout errors keep their class: Both HTTP clients raise litellm.Timeout before the guard runs. Timeout is not a BaseLLMException, so the existing error mapping returns it unchanged

Summary

Moves /v1/decisions through provider configs and the shared HTTP handler while keeping the Jev request and response shape

Adds provider configs and translation cases for all five supported providers
Preserves APIConnectionError for unreachable decisions deployments and fixes OpenRouter connection-error mapping
Expands connection-failure tests across providers and SDK calls
The numbered fixture-comment finding is fixed. The earlier HTTP reply-details finding was withdrawn by greptile-apps[bot] because preserving those details requires a shared error-contract change

_{Reviews (5) · Last reviewed commit: "fix(decisions): keep APIConnectionError ..." · Reviewed by Greptile}

codspeed · 2026-10-07T19:02:11Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_decisions_jev_provider_migration (bd81e17) with main (ed9c02d)}

codecov · 2026-10-07T19:11:44Z

Codecov Report

❌ Patch coverage is 95.72193% with 8 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...itellm/llms/cloudflare/decisions/transformation.py	81.25%	3 Missing ⚠️
litellm/decisions/main.py	96.00%	2 Missing ⚠️
litellm/llms/custom_httpx/llm_http_handler.py	92.00%	2 Missing ⚠️
...itellm/llms/perplexity/decisions/transformation.py	85.71%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

litellm-stampbot · 2026-10-07T19:12:24Z

@greptileai

litellm-stampbot · 2026-10-07T19:12:25Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…and the shared HTTP handler Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…ror and drop explanatory docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mateo-berri · 2026-10-08T00:48:24Z

@greptileai

…n every provider `_map_upstream_exception` now maps a status `_handle_error` synthesized for a non-HTTP failure to `APIConnectionError` before `exception_type` runs, so a Perplexity (openai-compatible) deployment that cannot be reached no longer answers `InternalServerError` where main answers `APIConnectionError`. Audit cells: the unreachable-deployment integration cell is parametrized over the five providers, OpenRouter connection failures are pinned on chat, stream, embeddings, decisions and the SDK, a deployment whose provider has no decisions config gets the 400 naming every supported provider, and a unit cell pins the mapper alone. The first docstring sentence of the five translation bases is dropped.

mateo-berri · 2026-10-08T02:23:15Z

@greptileai

mateo-berri · 2026-10-08T02:23:16Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit bd81e17. Configure here.}

mateo-berri · 2026-10-08T02:37:31Z

@greptileai

mateo-berri

LGTM

…and the shared HTTP handler (BerriAI#45130) * refactor(decisions): dispatch /v1/decisions through provider configs and the shared HTTP handler Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): map OpenRouter connection failures to APIConnectionError and drop explanatory docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(decisions): keep APIConnectionError for unreachable deployments on every provider `_map_upstream_exception` now maps a status `_handle_error` synthesized for a non-HTTP failure to `APIConnectionError` before `exception_type` runs, so a Perplexity (openai-compatible) deployment that cannot be reached no longer answers `InternalServerError` where main answers `APIConnectionError`. Audit cells: the unreachable-deployment integration cell is parametrized over the five providers, OpenRouter connection failures are pinned on chat, stream, embeddings, decisions and the SDK, a deployment whose provider has no decisions config gets the 400 naming every supported provider, and a unit cell pins the mapper alone. The first docstring sentence of the five translation bases is dropped. --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> (cherry picked from commit 0e09e99) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot assigned litellm-stampbot Oct 7, 2026

devin-ai-integration Bot added this pull request to stack #45131 October 7, 2026 18:51

This was referenced Oct 7, 2026

fix(decisions)!: refuse safety_identifier on providers that cannot take it unless drop_params drops it #44955

Merged

feat(decisions): add openai/gpt-6-luna through OpenAIDecisionsConfig #44969

Closed

cursor Bot reviewed Oct 7, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

Comment thread litellm/llms/custom_httpx/llm_http_handler.py

Comment thread litellm/llms/base_llm/decisions/transformation.py

Comment thread litellm/llms/base_llm/decisions/transformation.py Outdated

cursor Bot reviewed Oct 7, 2026

View reviewed changes

devin-ai-integration Bot force-pushed the litellm_decisions_jev_provider_migration branch 2 times, most recently from 115200e to a717565 Compare October 7, 2026 21:10

devin-ai-integration Bot force-pushed the litellm_decisions_jev_provider_migration branch 2 times, most recently from e3fda06 to d85c20e Compare October 7, 2026 23:05

Base automatically changed from litellm_decisions_openai_spec to main October 8, 2026 00:42

litellm-stampbot and others added 2 commits October 7, 2026 17:42

refactor(decisions): dispatch /v1/decisions through provider configs …

956d4c0

…and the shared HTTP handler Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

fix(decisions): map OpenRouter connection failures to APIConnectionEr…

5276719

…ror and drop explanatory docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mateo-berri force-pushed the litellm_decisions_jev_provider_migration branch from d85c20e to 5276719 Compare October 8, 2026 00:42

greptile-apps Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread tests/integration/translation/decisions/bases/cloudflare.py Outdated

cursor Bot reviewed Oct 8, 2026

View reviewed changes

mateo-berri approved these changes Oct 8, 2026

View reviewed changes

mateo-berri merged commit 0e09e99 into main Oct 8, 2026
109 of 112 checks passed

mateo-berri deleted the litellm_decisions_jev_provider_migration branch October 8, 2026 02:45

mateo-berri mentioned this pull request Oct 8, 2026

feat(decisions): add OpenAI as a Decisions provider behind a shared decisions format #45214

Merged

4 of 6 tasks

Conversation

devin-ai-integration Bot commented Oct 7, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Before (79f62db, 2 workers, 127.0.0.1:58201)

After (bd81e17, 2 workers, 127.0.0.1:52979)

Outbound wire

Spend rows

SDK client

/audit

Type

Caveats (if any)

Low

Final Attestation

Uh oh!

devin-ai-integration Bot commented Oct 7, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

litellm-stampbot commented Oct 7, 2026

Uh oh!

litellm-stampbot commented Oct 7, 2026

Uh oh!

CLAassistant commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

codecov Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

litellm-stampbot commented Oct 7, 2026

Uh oh!

litellm-stampbot commented Oct 7, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented Oct 8, 2026

Uh oh!

Uh oh!

mateo-berri commented Oct 8, 2026

Uh oh!

mateo-berri commented Oct 8, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented Oct 8, 2026

Uh oh!

mateo-berri left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

devin-ai-integration Bot commented Oct 7, 2026 •

edited by mateo-berri

Loading

Before (`79f62db`, 2 workers, 127.0.0.1:58201)

After (`bd81e17`, 2 workers, 127.0.0.1:52979)

devin-ai-integration Bot commented Oct 7, 2026 •

edited by mateo-berri

Loading

CLAassistant commented Oct 7, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading

codspeed Bot commented Oct 7, 2026 •

edited

Loading

codecov Bot commented Oct 7, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading