feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 by devin-ai-integration[bot] · Pull Request #42595 · BerriAI/litellm

devin-ai-integration · 2026-09-22T21:36:41Z

TLDR

Problem this solves:

TypeSafe Jev (classifier, passthroughs, guardrail) only exists from the 1.103 rc on
A customer runs the 1.103 rc in prod to get it and wants a stable line instead

How it solves it:

Cherry-picks the seven Jev PRs from main onto stable/1.102.x for v1.102.1
One prerequisite commit carries the pieces of main 109ca70f66 the picks build on
One dashboard commit adapts the feat(auto-router): add JEV classifier alongside LLM classifier #41886 UI pieces that referenced feat(ui): configure capability and Fuse v2 classifiers #41315 (not in the set)
No version bump here, stable/1.102.x already carries the v1.102.1 bump (fix(anthropic): backport #42152 and #42288 to stable/1.102.x for v1.102.1 #42538)

Included PRs, one commit per PR, cherry-picked with -m 1 from the main merge commit (plain pick for the #42301 squash), original author kept:

feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking #41607 feat(typesafe): TypeSafe Jev passthrough with logging and cost tracking (main deb9d8aedd, here 3b483c80f6)
feat(router): add TypeSafe Jev as a complexity router classifier #41615 feat(router): add TypeSafe Jev as a complexity router classifier (cf42b607c3, here b5ac61374e)
fix(proxy): forward every method on the typesafe pass-through route #41723 fix(proxy): forward every method on the typesafe pass-through route (34718f0da6, here 79b72b8594)
feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail #41757 feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail (2edda5aec3, here 4b0ffcdf41)
fix(proxy): enforce virtual key budgets for JEV test routing #41879 fix(proxy): enforce virtual key budgets for JEV test routing (1e161f516c, here 92a2a2f924)
feat(auto-router): add JEV classifier alongside LLM classifier #41886 feat(auto-router): add JEV classifier alongside LLM classifier (a83773cfa5, here e20cfbeaeb)
feat(openrouter): price typesafe/jev-1.13 and add an openrouter decisions pass-through #42301 feat(openrouter): price typesafe/jev-1.13 and add an openrouter decisions pass-through (1106b16745, here d0687347a7)

Two more commits on the branch:

1a3cec8f5e, ahead of feat(router): add TypeSafe Jev as a complexity router classifier #41615: the pieces of main 109ca70f66 (allow opted-in team members to manage their routers) that feat(router): add TypeSafe Jev as a complexity router classifier #41615 and fix(proxy): enforce virtual key budgets for JEV test routing #41879 build on: the DatabaseClient protocol and AUTO_ROUTER_MANAGE permission, the prisma_client kwarg on can_key_call_model / can_team_access_model, the lazy team_membership params on _check_team_member_model_access (keeping this line's ProxyException, since ModelAccessDeniedProxyException does not exist here), the member dry-run plumbing in auto_router_endpoints.py, and the matching test helpers
4bfac88281 fix(ui): adapt the jev dashboard pieces to stable/1.102.x. The feat(auto-router): add JEV classifier alongside LLM classifier #41886 dashboard files imported ./forecast_classifier_config and ./AutoRouterClassifierTabs from feat(ui): configure capability and Fuse v2 classifiers #41315, which is not on this line, so classifier_type_transition.ts is wired into ClassificationMethodConfig directly, classifier_llm_config is only serialized for LLM-backed classifiers (as on main), "hybrid" joins the classifier type union, heuristic_v2_success_threshold (not in this line's preset types) is dropped, and the jev connection-test params are hoisted into a named const so the dashboard lint budget stays at 551
d1cd2dccaf chore(backport): regenerate the openapi snapshot and dashboard api types for stable/1.102.x. The feat(router): add TypeSafe Jev as a complexity router classifier #41615 and feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail #41757 picks brought main's committed _lazy_openapi_snapshot.json and schema.d.ts verbatim, so they described classifier types, routing causes, classifier_* fields and a guardrail this line does not ship and lacked /auto_router/manage; both were regenerated under Python 3.12 (the interpreter check-ui-api-types uses). Generated artifacts only, no runtime change

Conflicts were resolved to what main has after each pick: #41615 in litellm/types/utils.py, complexity_router.py and config.py (this line lacks the capability and llm_v2 features, so the Jev hunks were applied on top of the line's code), auto_router_permissions.py and its test taken from main since they do not exist here. #41879 in auto_router_endpoints.py and its test around the member preview plumbing. #41886 in three python test files (formatting-only hunks kept as on the line, Jev additions applied) and about twelve dashboard files; StoredComplexityRouterConfig still lives in edit_auto_router_modal.tsx on this line, so the jev and per-turn fields went there. #42301 in gateway/routes/allowlist.py (added /openrouter/), success_handler.py (the typesafe branch also matches the openrouter decisions route) and _lazy_openapi_snapshot.json (only the added openrouter path block). schema.d.ts and the openapi snapshot only received the hunks each pick adds, nothing regenerated

Tests: every tests/test_litellm file the picks touch was run in the worktree, 1994 passed. The 14 requires_semantic_router tests in test_complexity_router.py fail on the line's tip before these picks too (semantic_router is not installed) and were left alone. The UI workflows do not run for stable/** bases, so the dashboard checks were run locally at 4bfac88281 under ui/litellm-dashboard: vitest on the touched test files (605 + 84 passed), eslint on every changed file (0 errors, local/no-large-inline-object-arg budget unchanged at 551), tsc --noEmit, npm run test:types, npm run build (the Dockerfile ui-builder stage), and make check at the repo root (PASS)

User Flow

Before: an admin on stable 1.102.x cannot route through TypeSafe Jev at all, so the only way to get it is running the 1.103 rc in prod

The admin starts the proxy from stable/1.102.x with TYPESAFE_API_KEY set and an auto router auto-jev whose complexity_router_config has "classifier_type": "jev"
A developer sends POST https://litellm-domain/v1/chat/completions with "model": "auto-jev" and gets 400 Invalid model name passed in model=auto-jev (the router dropped that deployment at boot)
The admin tries POST https://litellm-domain/auto_router/test_routing with "classifier_type": "jev" and gets 422 Input should be 'heuristic', 'heuristic_v2', 'llm', 'custom', 'heuristic_first' or 'hybrid'
They try POST https://litellm-domain/model/new with the same jev config and get 400 with the same message
The developer sends POST https://litellm-domain/typesafe/v1/systemone with a jev-latest payload and gets 404 {"detail":"Not Found"}; POST https://litellm-domain/openrouter/alpha/decisions is 404 too
On https://litellm-domain/ui/models-and-endpoints, Auto-Routers tab, Add Auto Router, the classification method picker shows Heuristic, Heuristic v2, LLM Classifier, Heuristic first and Hybrid, nothing for JEV
The admin upgrades prod to v1.103.0-rc.1 to get any of this

After: the same admin gets the JEV classifier, the TypeSafe passthroughs and the compaction guardrail on stable 1.102.x (v1.102.1)

The admin starts the proxy from this branch with the same config
The developer sends POST https://litellm-domain/v1/chat/completions with "model": "auto-jev" and gets 200; the x-litellm-complexity-router-tier header says SIMPLE for the short prompt and COMPLEX for the long one, with x-litellm-complexity-router-cause: jev_classifier; POST https://litellm-domain/v1/messages and POST https://litellm-domain/v1/responses with "model": "auto-jev" route the same way
The admin sends POST https://litellm-domain/auto_router/test_routing with "classifier_type": "jev" and gets 200 with routed_model, tier and the jev-classifier:SIMPLE / jev-confidence=... signals; with a key over budget the call is 400 budget_exceeded instead of a free dry run
POST https://litellm-domain/model/new with the jev config is 200 and chats with "model": "db-auto-jev" route the same way
POST https://litellm-domain/typesafe/v1/systemone is forwarded and comes back 200 with TypeSafe's answers, PUT is forwarded too (405 from TypeSafe), a bad key gets 401 LiteLLM Virtual Key expected; POST https://litellm-domain/openrouter/alpha/decisions with "model": "typesafe/jev-1.13" is 200 with "model":"typesafe/jev-1.13-20260917"
The same UI picker now offers JEV Classifier with JEV Model and JEV Timeout (ms) fields
A chat sent with "guardrails": ["typesafe-compaction"] over a long tool conversation is billed at 962 prompt tokens instead of 1119 for the same chat without it
The admin stays on stable 1.102.x instead of the rc

Relevant issues

Backport of #41607, #41615, #41723, #41757, #41879, #41886 and #42301 to stable/1.102.x

Affected release

Linear ticket

Resolves LIT-8369

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two proxies from the same config, each with its own empty Postgres, real provider calls (Anthropic, TypeSafe, OpenRouter) with real spend. Before is the stable/1.102.x merge base 95293834e8 on port 16390, After is the PR tip 4bfac88281 on port 16391 (its process runs the Python tree of d0687347a7, which is byte-identical to the tip outside ui/litellm-dashboard; the dashboard screenshots come from the bundle built at 4bfac88281). Both started with

python litellm/proxy/proxy_cli.py --config config.yaml --port <port> --num_workers 2 --detailed_debug

config.yaml (LITELLM_MASTER_KEY, DATABASE_URL, ANTHROPIC_API_KEY, TYPESAFE_API_KEY in the environment):

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
litellm_settings:
  drop_params: true
model_list:
  - model_name: claude-haiku-4-5
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: auto-llm
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_default_model: claude-sonnet-5
      complexity_router_config:
        tiers: {SIMPLE: claude-haiku-4-5, MEDIUM: claude-sonnet-5, COMPLEX: claude-sonnet-5, REASONING: claude-sonnet-5}
        classifier_type: llm
        classifier_llm_config: {model: claude-haiku-4-5, timeout_ms: 8000}
  - model_name: auto-heuristic
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_default_model: claude-sonnet-5
      complexity_router_config:
        tiers: {SIMPLE: claude-haiku-4-5, MEDIUM: claude-sonnet-5, COMPLEX: claude-sonnet-5, REASONING: claude-sonnet-5}
        classifier_type: heuristic
  - model_name: auto-jev
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_default_model: claude-sonnet-5
      complexity_router_config:
        tiers: {SIMPLE: claude-haiku-4-5, MEDIUM: claude-sonnet-5, COMPLEX: claude-sonnet-5, REASONING: claude-sonnet-5}
        classifier_type: jev
        jev_classifier_config: {model: jev-latest, timeout_ms: 8000}
guardrails:
  - guardrail_name: typesafe-compaction
    litellm_params:
      guardrail: typesafe
      mode: pre_call
      default_on: false
      api_key: os.environ/TYPESAFE_API_KEY

Keys used below, generated the same way on both sides: $MASTER_KEY is the config's master key; $KEY comes from POST /key/generate {"key_alias":"ab-plain","max_budget":10} (an internal-user key); $ADMIN_KEY and $ADMIN_KEY_EXHAUSTED are keys of a proxy_admin user created with POST /user/new {"user_id":"ab-admin","user_role":"proxy_admin","auto_create_key":false}, generated with max_budget 10 and max_budget 0 ("metadata":{"throttle_on_budget_exceeded":true}) respectively. $BASE is http://localhost:16390 for Before and http://localhost:16391 for After. The compaction case sends this 9-message tool conversation:

The /v1/messages and /v1/responses cases ran last, against the same two trees booted from the same config minus database_url (ports 17490 and 17491, master key only), after the databases above were torn down

tool conversation payload

{
  "model": "claude-haiku-4-5",
  "max_tokens": 60,
  "guardrails": [
    "typesafe-compaction"
  ],
  "messages": [
    {
      "role": "system",
      "content": "You are a deployment assistant."
    },
    {
      "role": "user",
      "content": "What is the weather in Paris?"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "c1",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"city\":\"Paris\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "c1",
      "content": "Paris: 18C, light rain, humidity 72 percent, wind 12 km/h from the west, UV index 2, sunrise 07:14, sunset 19:52, pollen low, air quality index 31, visibility 9 km, pressure 1013 hPa, dew point 13C, cloud cover 80 percent, chance of rain 60 percent through the evening."
    },
    {
      "role": "user",
      "content": "And the ACME stock price?"
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "c2",
          "type": "function",
          "function": {
            "name": "get_quote",
            "arguments": "{\"ticker\":\"ACME\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "c2",
      "content": "ACME: 142.17 USD, change +1.3 percent, open 140.02, high 143.10, low 139.80, volume 2.1M, market cap 48B, PE 31.2, dividend yield 0.8 percent, 52 week high 155.00, 52 week low 98.40, next earnings 2026-10-21, analyst consensus hold, average target 150.00."
    },
    {
      "role": "user",
      "content": "Show me the deployment runbook for the payments service."
    },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        {
          "id": "c3",
          "type": "function",
          "function": {
            "name": "get_runbook",
            "arguments": "{\"service\":\"payments\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "c3",
      "content": "Payments runbook: 1) freeze deploys in #payments-ops, 2) run db migration payments-2026-09 with --dry-run first, 3) scale canary to 5 percent, 4) watch p99 latency and error rate for 15 minutes, 5) if error rate exceeds 0.5 percent roll back with deploy rollback payments, 6) otherwise promote to 100 percent and unfreeze, 7) post the summary in #payments-ops with the deploy id."
    },
    {
      "role": "user",
      "content": "What is step 5 of the runbook?"
    }
  ]
}

Before (`9529383`, port 16390)

Chat through the JEV auto router

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/chat/completions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","messages":[{"role":"user","content":"What is the capital of France? Answer in one word."}],"max_tokens":40}'

HTTP 400, {"error":{"message":"/chat/completions: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provider_specific_f...
Same call with the long design prompt ("Design a distributed rate limiter for a multi-region API gateway...")
HTTP 400, {"error":{"message":"/chat/completions: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provider_specific_f...

Messages and Responses through the JEV auto router

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/messages \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","max_tokens":40,"messages":[{"role":"user","content":"What is the capital of France? Answer in one word."}]}'

HTTP 400, {"type":"error","error":{"type":"invalid_request_error","message":"anthropic_messages: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","provider_...
Same call with the long design prompt and max_tokens 200
HTTP 400, {"type":"error","error":{"type":"invalid_request_error","message":"anthropic_messages: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","provider_...

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/responses \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","max_output_tokens":40,"input":"What is the capital of France? Answer in one word."}'

HTTP 400, {"error":{"message":"/responses: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provid...
Same call with the long design prompt and max_output_tokens 200
HTTP 400, {"error":{"message":"/responses: Invalid model name passed in model=auto-jev. Call /v1/models to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provid...

Dry-run routing with the JEV classifier

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/auto_router/test_routing \
     -H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
     -d '{"prompt":"What is the capital of France? Answer in one word.","complexity_router_config":{"tiers":{"SIMPLE":"claude-haiku-4-5","MEDIUM":"claude-sonnet-5","COMPLEX":"claude-sonnet-5","REASONING":"claude-sonnet-5"},"classifier_type":"jev","jev_classifier_config":{"model":"jev-latest"}}}'

HTTP 422, {"detail":[{"type":"literal_error","loc":["body","complexity_router_config","classifier_type"],"msg":"Input should be 'heuristic', 'heuristic_v2', 'llm', 'custom', 'heuristic_first' or 'hybrid'","input":"jev","ctx":{"expected":"'heuristic', 'heuristic_v2', 'll...
Same call with $ADMIN_KEY_EXHAUSTED (max_budget 0)
HTTP 422, {"detail":[{"type":"literal_error","loc":["body","complexity_router_config","classifier_type"],"msg":"Input should be 'heuristic', 'heuristic_v2', 'llm', 'custom', 'heuristic_first' or 'hybrid'","input":"jev","ctx":{"expected":"'heuristic', 'heuristic_v2', 'll...
Same call with the master key
HTTP 422
Same call with $KEY (a plain internal-user key)
HTTP 422, {"detail":[{"type":"literal_error","loc":["body","complexity_router_config","classifier_type"],"msg":"Input should be 'heuristic', 'heuristic_v2', 'llm', 'custom', 'heuristic_first' or 'hybrid'","input":"jev","ctx":{"expected":"'heuristic',...

TypeSafe passthrough

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/typesafe/v1/systemone \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"jev-latest","state":"The sky is blue on a clear day.","questions":{"blue":{"type":"noul","instructions":"Is the sky described as blue?"}}}'

HTTP 404, {"detail":"Not Found"}
PUT on the same route with {"state":"x"}
HTTP 404, {"detail":"Not Found"}
Same POST with Authorization: Bearer not-a-key
HTTP 404, {"detail":"Not Found"}

Same POST with odd payloads:

payload	HTTP	body
`"state": 7`	404	`{"detail":"Not Found"}`
`"state": ["a","b"]`	404	`{"detail":"Not Found"}`
`"state": ""`	404	`{"detail":"Not Found"}`
a 5 KB `state`	404	`{"detail":"Not Found"}`
the step 1 payload again	404	`{"detail":"Not Found"}`

OpenRouter decisions passthrough

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/openrouter/alpha/decisions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"typesafe/jev-1.13","state":"The sky is blue on a clear day.","questions":{"blue":{"type":"noul","instructions":"Is the sky described as blue?"}}}'

HTTP 404, {"detail":"Not Found"}

TypeSafe compaction guardrail

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/chat/completions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"claude-haiku-4-5","max_tokens":60,"guardrails":["typesafe-compaction"],"messages":<the 9-message tool conversation from the setup>}'

HTTP 200, usage.prompt_tokens 1119
Same conversation without guardrails
HTTP 200, usage.prompt_tokens 1119

JEV auto router created through the API

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/model/new \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model_name":"db-auto-jev","litellm_params":{"model":"auto_router/complexity_router","complexity_router_default_model":"claude-sonnet-5","complexity_router_config":{"tiers":{"SIMPLE":"claude-haiku-4-5","MEDIUM":"claude-sonnet-5","COMPLEX":"claude-sonnet-5","REASONING":"claude-sonnet-5"},"classifier_type":"jev","jev_classifier_config":{"model":"jev-latest"}}}}'

HTTP 400, {"error":{"message":"complexity_router_config is invalid at classifier_type: Input should be 'heuristic', 'heuristic_v2', 'llm', 'custom', 'heuristic_first' or 'hybrid'. The router would drop this deployment at load time, so the write is rejected instead.","type":"validation_error","param":"litellm_...
POST $BASE/v1/chat/completions with "model": "db-auto-jev" and the capital-of-France prompt
HTTP 400, {"error":{"message":"/chat/completions: Invalid model name passed in model=db-auto-jev. Call /v1/models to view available models for your key.","type":"invalid_request_error","param":null,"code":"400","provider_specifi...

TypeSafe guardrail created through the API

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/guardrails \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"guardrail":{"guardrail_name":"typesafe-db","litellm_params":{"guardrail":"typesafe","mode":"pre_call","default_on":false,"api_key":"os.environ/TYPESAFE_API_KEY"}}}'

HTTP 500, {"detail":"400: Guardrail configuration error: Unsupported guardrail: typesafe"}

Admin UI classifier picker

Open $BASE/ui/models-and-endpoints, click the Auto-Routers tab, click Add Auto Router, expand Detailed Configuration, then Advanced: Classification Method
The picker offers Heuristic, Heuristic v2, LLM Classifier, Heuristic first and Hybrid; there is no JEV option

Existing surfaces (regression sweep)

Each row below is one curl of the form used above (same $BASE, the key named in the row, the same simple and hard prompts), run in the listed order on this side

Observed, one row per call:

call	HTTP	detail
POST /v1/chat/completions `auto-llm`, simple prompt	200	model `auto-llm`, tier SIMPLE
POST /v1/chat/completions `auto-llm`, hard prompt	200	model `auto-llm`, tier COMPLEX
POST /v1/chat/completions `auto-heuristic`	200	model `auto-heuristic`, tier SIMPLE
POST /v1/chat/completions `claude-haiku-4-5`	200	model `claude-haiku-4-5`
POST /v1/messages `claude-haiku-4-5`	200	model `claude-haiku-4-5`
POST /v1/responses `auto-llm`	200	model `auto-llm`, tier SIMPLE
POST /v1/chat/completions with the max_budget 0 key	429	`{"error":{"message":"Budget has been exceeded! Key=ab-exhausted (sk-...) Current cost:...`
POST /auto_router/test_routing llm, admin key	200
POST /auto_router/test_routing heuristic, admin key	200
POST /auto_router/test_routing llm, exhausted admin key	400	`{"error":{"message":"Budget has been exceeded! Key=ab-admin-exhausted (sk-...) Current...`
POST /auto_router/test_routing heuristic, exhausted admin key	200
POST /auto_router/test_routing llm, master key	200
POST /auto_router/test_routing llm, internal-user key	403	`{"detail":{"error":"User does not have permission to dry-run an auto router. Your role=int...`
POST /auto_router/test_routing heuristic, internal-user key	403	`{"detail":{"error":"User does not have permission to dry-run an auto router. Your role=int...`
POST /model/new `db-auto-llm` (llm classifier)	200
POST /v1/chat/completions `db-auto-llm`	200	model `db-auto-llm`, tier SIMPLE
GET /model/info	200
GET /v1/models	200
GET /model_group/info	200
GET /guardrails/list	200
GET /guardrails/ui/provider_specific_params	200
POST /user/new (proxy_admin user)	200
GET /health	200
GET /health/readiness	200
GET /health after all of the above	200

After (`4bfac88`, port 16391)

The tip is d1cd2dccaf; it adds only the regenerated OpenAPI snapshot and the dashboard .d.ts types on top of 4bfac88281, neither of which is executed on any route or screen below, so this run stands for the tip

Chat through the JEV auto router

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/chat/completions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","messages":[{"role":"user","content":"What is the capital of France? Answer in one word."}],"max_tokens":40}'

HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: SIMPLE, x-litellm-complexity-router-cause: jev_classifier, prompt_tokens 19 completion_tokens 4
Same call with the long design prompt ("Design a distributed rate limiter for a multi-region API gateway...")
HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: COMPLEX, x-litellm-complexity-router-cause: jev_classifier, prompt_tokens 67 completion_tokens 6903

Messages and Responses through the JEV auto router

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/messages \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","max_tokens":40,"messages":[{"role":"user","content":"What is the capital of France? Answer in one word."}]}'

HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: SIMPLE, x-litellm-complexity-router-cause: jev_classifier, input_tokens 19 output_tokens 4
Same call with the long design prompt and max_tokens 200
HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: COMPLEX, x-litellm-complexity-router-cause: jev_classifier, input_tokens 67 output_tokens 4906

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/responses \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model":"auto-jev","max_output_tokens":40,"input":"What is the capital of France? Answer in one word."}'

HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: SIMPLE, x-litellm-complexity-router-cause: jev_classifier, input_tokens 19 output_tokens 4
Same call with the long design prompt and max_output_tokens 200
HTTP 200, model auto-jev, header x-litellm-complexity-router-tier: COMPLEX, x-litellm-complexity-router-cause: jev_classifier, input_tokens 67 output_tokens 6586

Dry-run routing with the JEV classifier

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/auto_router/test_routing \
     -H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
     -d '{"prompt":"What is the capital of France? Answer in one word.","complexity_router_config":{"tiers":{"SIMPLE":"claude-haiku-4-5","MEDIUM":"claude-sonnet-5","COMPLEX":"claude-sonnet-5","REASONING":"claude-sonnet-5"},"classifier_type":"jev","jev_classifier_config":{"model":"jev-latest"}}}'

HTTP 200, {"routed_model":"claude-haiku-4-5","tier":"SIMPLE","cause":"jev_classifier","classifier_model":"typesafe/jev-1.13.0","signals":["jev-classifier:SIMPLE","jev-confidence=1.000000","tier-probability:REASONING=0.000000","tier-probability:MEDIUM=0.000000","tier-probability:SIMPLE=1.000000","tier-probability:COMPLEX=0.000000"]}
Same call with $ADMIN_KEY_EXHAUSTED (max_budget 0)
HTTP 400, {"error":{"message":"Budget has been exceeded! Key=ab-admin-exhausted (sk-...) Current cost: 0.0, Max budget: 0.0","type":"budget_exceeded","param":null,"code":"400"}}
Same call with the master key
HTTP 200
Same call with $KEY (a plain internal-user key)
HTTP 403, {"detail":{"error":"User does not have permission to dry-run an auto router. Your role=internal_user. Call as a PROXY_ADMIN, or as a team admin by specifying a team_id."}}

TypeSafe passthrough

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/typesafe/v1/systemone \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"jev-latest","state":"The sky is blue on a clear day.","questions":{"blue":{"type":"noul","instructions":"Is the sky described as blue?"}}}'

HTTP 200, {"model":"jev-1.13.0","answers":{"blue":{"type":"noul","noul":0.99}},"usage":{"input_tokens":282,"output_tokens":20}}
PUT on the same route with {"state":"x"}
HTTP 405, {"detail":"Method Not Allowed"}
Same POST with Authorization: Bearer not-a-key
HTTP 401, {"error":{"message":"LiteLLM Virtual Key expected. Received=not-****-key, expected to start with 'sk-'.","type":"auth_error","param":"None","code":"401"}}

Same POST with odd payloads:

payload	HTTP	body
`"state": 7`	422	`{"detail":[{"type":"string_type","loc":["body","state","str"],"msg":"Input should be a valid string","input":7...`
`"state": ["a","b"]`	200	`{"model":"jev-1.13.0","answers":{"q":{"type":"noul","noul":0.97}},"usage":{"input_tokens":278,"output_tokens":...`
`"state": ""`	200	`{"model":"jev-1.13.0","answers":{"q":{"type":"noul","noul":0.8}},"usage":{"input_tokens":270,"output_tokens":2...`
a 5 KB `state`	200	`{"model":"jev-1.13.0","answers":{"q":{"type":"noul","noul":0.96}},"usage":{"input_tokens":1832,"output_tokens"...`
the step 1 payload again	200	`{"model":"jev-1.13.0","answers":{"blue":{"type":"noul","noul":0.99}},"usage":{"input_tokens":282,"output_token...`

OpenRouter decisions passthrough

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/openrouter/alpha/decisions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"typesafe/jev-1.13","state":"The sky is blue on a clear day.","questions":{"blue":{"type":"noul","instructions":"Is the sky described as blue?"}}}'

HTTP 200, {"model":"typesafe/jev-1.13-20260917","answers":{"blue":{"type":"noul","noul":0.99}},"usage":{"input_tokens":282,"output_tokens":20,"cost":1.1844e-05},"id":"gen-dec-1790127449-uSzKbgQH085seSsxNoWK","provider":"TypeSafe"}

TypeSafe compaction guardrail

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/v1/chat/completions \
     -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
     -d '{"model":"claude-haiku-4-5","max_tokens":60,"guardrails":["typesafe-compaction"],"messages":<the 9-message tool conversation from the setup>}'

HTTP 200, usage.prompt_tokens 962
Same conversation without guardrails
HTTP 200, usage.prompt_tokens 1119

JEV auto router created through the API

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/model/new \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"model_name":"db-auto-jev","litellm_params":{"model":"auto_router/complexity_router","complexity_router_default_model":"claude-sonnet-5","complexity_router_config":{"tiers":{"SIMPLE":"claude-haiku-4-5","MEDIUM":"claude-sonnet-5","COMPLEX":"claude-sonnet-5","REASONING":"claude-sonnet-5"},"classifier_type":"jev","jev_classifier_config":{"model":"jev-latest"}}}}'

HTTP 200, model_id c746f2c7-6a20-4d9b-8536-5d3baab25654
POST $BASE/v1/chat/completions with "model": "db-auto-jev" and the capital-of-France prompt
HTTP 200, model db-auto-jev, header x-litellm-complexity-router-tier: SIMPLE, x-litellm-complexity-router-cause: jev_classifier, prompt_tokens 19 completion_tokens 4

TypeSafe guardrail created through the API

curl -s -w '\nHTTP %{http_code}\n' -X POST $BASE/guardrails \
     -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
     -d '{"guardrail":{"guardrail_name":"typesafe-db","litellm_params":{"guardrail":"typesafe","mode":"pre_call","default_on":false,"api_key":"os.environ/TYPESAFE_API_KEY"}}}'

HTTP 200, guardrail_name typesafe-db (body holds the guardrail id and params, not shown)

Admin UI classifier picker

Open $BASE/ui/models-and-endpoints, click the Auto-Routers tab, click Add Auto Router, expand Detailed Configuration, then Advanced: Classification Method
The picker adds JEV Classifier ("uses TypeSafe System One Choice to decide the tier")
Selecting it shows the JEV Model (jev-latest) and JEV Timeout (ms) (3000) fields plus the classifier circuit breaker toggle

Existing surfaces (regression sweep)

Each row below is one curl of the form used above (same $BASE, the key named in the row, the same simple and hard prompts), run in the listed order on this side

Observed, one row per call:

call	HTTP	detail
POST /v1/chat/completions `auto-llm`, simple prompt	200	model `auto-llm`, tier SIMPLE
POST /v1/chat/completions `auto-llm`, hard prompt	200	model `auto-llm`, tier COMPLEX
POST /v1/chat/completions `auto-heuristic`	200	model `auto-heuristic`, tier SIMPLE
POST /v1/chat/completions `claude-haiku-4-5`	200	model `claude-haiku-4-5`
POST /v1/messages `claude-haiku-4-5`	200	model `claude-haiku-4-5`
POST /v1/responses `auto-llm`	200	model `auto-llm`, tier SIMPLE
POST /v1/chat/completions with the max_budget 0 key	429	`{"error":{"message":"Budget has been exceeded! Key=ab-exhausted (sk-...) Current cost:...`
POST /auto_router/test_routing llm, admin key	200
POST /auto_router/test_routing heuristic, admin key	200
POST /auto_router/test_routing llm, exhausted admin key	400	`{"error":{"message":"Budget has been exceeded! Key=ab-admin-exhausted (sk-...) Current...`
POST /auto_router/test_routing heuristic, exhausted admin key	200
POST /auto_router/test_routing llm, master key	200
POST /auto_router/test_routing llm, internal-user key	403	`{"detail":{"error":"User does not have permission to dry-run an auto router. Your role=int...`
POST /auto_router/test_routing heuristic, internal-user key	403	`{"detail":{"error":"User does not have permission to dry-run an auto router. Your role=int...`
POST /model/new `db-auto-llm` (llm classifier)	200
POST /v1/chat/completions `db-auto-llm`	200	model `db-auto-llm`, tier SIMPLE
GET /model/info	200
GET /v1/models	200
GET /model_group/info	200
GET /guardrails/list	200
GET /guardrails/ui/provider_specific_params	200
POST /user/new (proxy_admin user)	200
GET /health	200
GET /health/readiness	200
GET /health after all of the above	200

Live PR risk

Verdict: no breaking or backward-incompatible change on an existing surface. Every call that existed before (25 in the sweep above, plus the delete-then-re-add of a DB auto router) returns the same status and the same routing tier on both sides

Breaking: none found. The base of every existing route, the config keys, the DB schema and the dashboard routes are unchanged; the picks only add the jev classifier literal, the typesafe guardrail, the /typesafe/* and /openrouter/* pass-through routes and the dashboard picker entry

Backward incompatible: POST /auto_router/test_routing with "classifier_type": "jev" moves from 422 (unknown literal) to the same permission and budget checks the other classifiers already get (403 for an internal-user key, 400 budget_exceeded for an exhausted key, 200 otherwise). Nothing that returned 200 before returns anything else now

Regression risk:

Delete-then-re-add of a DB auto router (POST /model/delete, then POST /model/new with the same name) works on both sides: delete 200, /model/info empty, re-add 200, chat 200. One head re-add during QA returned 500 not live in this pod's router after the reload; the cause was orphan DB rows created while the rig was overloaded (rows saved to the DB but refused by the router's duplicate-name guard, which holds the name until the orphan is deleted). litellm/router.py is byte-identical between the merge base and the tip and the guard exists on main, so it is not something this PR changes
Spend logging for the new routes was read back from LiteLLM_SpendLogs at the tip: typesafe/jev-1.13.0 and openrouter/typesafe/jev-1.13-20260917 rows carry token counts and non-zero spend, and the classifier call of every JEV-routed chat is its own row next to the routed model's row

Dependency graph:

tested: complexity router classifier dispatch (jev next to llm and heuristic), test_routing permission and budget path, /model/new validation of complexity_router_config, guardrail registry (typesafe), pass-through route registration and auth (/typesafe/*, /openrouter/alpha/decisions), pass-through spend logging, the dashboard add-auto-router modal
untested: streaming through a JEV router, the edit-auto-router modal, assigning the typesafe guardrail to a key or team, the other three lines (their own PRs)
verified live: Anthropic, TypeSafe and OpenRouter upstreams, all with real spend
unreachable here: a rolling upgrade from a running v1.102.0 with existing DB auto routers and keys; the schema is unchanged and the config types only gain a literal, so an old row loads the same way, but it was not simulated

Not verified: streaming through a JEV router, the edit modal, rolling upgrade

Type

🆕 New Feature

Caveats (if any)

Medium

The UI workflows do not run for stable/** bases, so the dashboard checks ran locally at the tip (vitest, eslint, tsc, test:types, npm run build, make check)
Streaming through a JEV router and the edit-auto-router modal were not driven; the code paths are the ones the picks carry from main unchanged

Low

Against the pre-pick base (95293834e8), scripts/type_discipline_gate.py counts LIT001 at 22188 vs the line's limit 22174 (+15, in the picked auth_checks.py and guardrail_hooks/typesafe/) and scripts/test_quality_gate.py counts TQ008 at 11151 vs 10993 (+73, in pick-modified test files). The code is main's verbatim, the workflows enforcing these budgets do not run for stable/** bases, and budget JSON files are never edited on a branch, so the counts are left as they are rather than refactoring main's code on a stable line
test_routing with an internal-user key is 403 before and after (needs PROXY_ADMIN or a team admin), so the JEV dry run has the same audience as the other classifiers
The 14 requires_semantic_router tests in test_complexity_router.py fail on this line with or without the picks (semantic_router is not installed here)
A complexity router's tier params override the caller's max_tokens (the hard prompts come back with thousands of completion tokens at max_tokens 40); this is how the router already behaves on the merge base with the llm classifier, not something the picks change
feat(ui): configure capability and Fuse v2 classifiers #41315 (forecast classifier and the classifier tabs) is not in the set; the two dashboard pieces of feat(auto-router): add JEV classifier alongside LLM classifier #41886 that imported it were adapted in 4bfac88281 instead of pulling it in

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/b6d5b6c20aec42babbd9213ec27ef3bd
Open in Devin Desktop: https://app.devin.ai/desktop/session/b6d5b6c20aec42babbd9213ec27ef3bd?variant=devin
Requested by: @mateo-berri

Note

Medium Risk
Broad additive changes across routing, auth, guardrails, and external TypeSafe/OpenRouter calls; existing routes stay compatible but misconfiguration or upstream outages can block or alter requests on new JEV paths.

Overview
This backport brings TypeSafe Jev onto the stable proxy line: new /typesafe/* and /openrouter/* pass-through routes (with token/cost logging), a typesafe guardrail that compacts stale tool results via Jev /v1/systemone, and classifier_type: "jev" on complexity auto-routers (tier choice, circuit breaker, classifier probabilities in routing logs).

Auto-router dry-runs and member-managed routers gain stricter gates: /auto_router/manage team permission, dependency/model checks for non-admins, optional saved_model_id on test routing, and virtual-key budget enforcement for JEV test routing. Model writes merge stored Jev credentials when patching configs; router health sweeps skip evaluation dependencies so Jev classifiers do not mark deployments unhealthy.

Pricing/schema updates add evaluation mode and Jev model entries; allowlists and lazy route loading register the new pass-through prefixes.

^{Reviewed by Cursor Bugbot for commit d1cd2dc. Bugbot is set up for automated code reviews on this repo. Configure here.}

Link to Devin session: https://app.devin.ai/sessions/dcedce8035d64a199713bae4b2b90b7f
Open in Devin Desktop: https://app.devin.ai/desktop/session/dcedce8035d64a199713bae4b2b90b7f?variant=devin

4bfac88 passes /live-pr-risk

…acking Backport of #41607 to stable/1.102.x. Cherry-picked from deb9d8a (main). Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration · 2026-09-22T21:36:44Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

greptile-apps · 2026-09-22T21:40:32Z

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding findings after the generated artifacts were synchronized to the stable branch

Findings

JEV Key Stored Unencrypted ▶

Summary

This PR backports TypeSafe JEV routing, passthrough, guardrail, pricing, permissions, and dashboard support to stable/1.102.x. The latest commit regenerates the OpenAPI snapshot and dashboard API types so they describe this stable branch rather than main-only functionality

Adds JEV as a complexity-router classifier with dry-run, budget, and permission handling
Adds authenticated TypeSafe and OpenRouter passthrough routes with spend logging
Adds TypeSafe conversation compaction and dashboard configuration support
Synchronizes generated OpenAPI and TypeScript definitions with the stable-line backend

_{Reviews (4) · Last reviewed commit: "chore(backport): regenerate the openapi ..."}

mateo-berri · 2026-09-22T21:43:04Z

bugbot run

devin-ai-integration · 2026-09-22T21:43:59Z

@greptileai

cursor

Stale Bugbot comment from a previous run.

Prerequisite for #41615 and #41879 on stable/1.102.x. Cherry-picked from 109ca70 (main).

Backport of #41615 to stable/1.102.x. Cherry-picked from cf42b60 (main).

Backport of #41723 to stable/1.102.x. Cherry-picked from 34718f0 (main).

Backport of #41757 to stable/1.102.x. Cherry-picked from 2edda5a (main).

Backport of #41879 to stable/1.102.x. Cherry-picked from 1e161f5 (main).

Backport of #41886 to stable/1.102.x. Cherry-picked from a83773c (main).

…ions pass-through (#42301) Backport of #42301 to stable/1.102.x. Cherry-picked from 1106b16 (main).

CLAassistant · 2026-09-22T23:00:46Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 6 committers have signed the CLA.

✅ yuneng-berri
✅ mateo-berri
✅ moe-berri
✅ tin-berri
❌ yassin-berriai
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

mateo-berri · 2026-09-23T01:57:24Z

bugbot run

mateo-berri · 2026-09-23T01:57:25Z

@greptileai

greptile-apps · 2026-09-23T02:01:08Z

        encrypted_params: Final = {
-            k: encrypt_value_helper(v) for k, v in updated_patch.litellm_params.model_dump(exclude_none=True).items()
+            k: (
+                _effective_complexity_router_config(updated_patch.litellm_params, db_model.litellm_params)
+                if k == "complexity_router_config"
+                else encrypt_value_helper(v)
+            )
+            for k, v in updated_patch.litellm_params.model_dump(exclude_none=True).items()


JEV Key Stored Unencrypted

When a caller supplies jev_classifier_config.api_key, this branch persists the complexity-router configuration without encrypting the nested key. The create path also encrypts only the outer mapping, leaving nested strings unchanged. Creating or updating a JEV router with an inline key therefore stores the provider credential in plaintext instead of protecting it like other model credentials.

How this was verified: The accepted nested API-key field flows unchanged through the model create and update serialization paths into the database JSON.

Knowledge Base Used: Management APIs and resources

Same lines as main since #41886: the backport carries them verbatim, and a stable line never redesigns persistence. Main fix tracked as LIT-8389

Understood. This persistence behavior is intentionally carried over from #41886, and redesigning nested JEV credential encryption is out of scope for this stable-line backport. I’ll treat the issue as deferred to the main-branch fix tracked by LIT-8389 rather than a blocker for this PR.

cursor

Stale Bugbot comment from a previous run.

cursor · 2026-09-23T02:03:03Z

+        return validated
+    except ValidationError as exc:
+        location: Final = ".".join(str(part) for part in exc.errors()[0]["loc"])
+        raise HTTPException(status_code=400, detail=f"Invalid member auto-router configuration at {location}.") from exc


Member saved JEV probe rejects credentials

Medium Severity

Member test_routing with saved_model_id revalidates the stored JEV config through _MemberJevClassifierConfig, which forbids api_key and api_base. A saved router that has TypeSafe credentials therefore fails member validation instead of using the server-side config.

Additional Locations (1)

litellm/proxy/management_endpoints/auto_router_endpoints.py#L491-L498

^{Reviewed by Cursor Bugbot for commit 4bfac88. Configure here.}

This file is byte-identical to main (#41615 and #41886 picked verbatim), so a saved router's member validation matches main. A stable line never redesigns it

…pes for stable/1.102.x The committed schema.d.ts and lazy OpenAPI snapshot were taken verbatim from main by the #41615 and #41757 picks, so they described classifier types, routing causes, and a guardrail that this line does not ship and lacked the /auto_router/manage permission. Regenerated under Python 3.12, the interpreter the check-ui-api-types workflow uses.

mateo-berri · 2026-09-23T02:17:24Z

@greptileai

mateo-berri · 2026-09-23T02:27:44Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d1cd2dc. Configure here.}

mateo-berri

LGTM

…02.1) (#736) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.102.0` → `v1.102.1` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.102.1`](https://github.com/BerriAI/litellm/releases/tag/v1.102.1) [Compare Source](BerriAI/litellm@v1.102.0...v1.102.1) ##### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.1/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** ##### What's Changed - fix(anthropic): backport [#42152](BerriAI/litellm#42152) and [#42288](BerriAI/litellm#42288) to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42538](BerriAI/litellm#42538) - feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42595](BerriAI/litellm#42595) - chore(release): backport [#42388](BerriAI/litellm#42388) and [#41462](BerriAI/litellm#41462) to stable/1.102.x by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42618](BerriAI/litellm#42618) **Full Changelog**: <BerriAI/litellm@v1.102.0...v1.102.1> </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/London) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.hayden.moe/hayden/phoebe/pulls/736

…02.1) (#2200) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.102.0` → `v1.102.1` | --- > ⚠️ **Warning** > > Some dependencies could not be looked up. Check the [Dependency Dashboard](issues/6) for more information. --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.102.1`](https://github.com/BerriAI/litellm/releases/tag/v1.102.1) [Compare Source](BerriAI/litellm@v1.102.0...v1.102.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.1/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - fix(anthropic): backport [#42152](BerriAI/litellm#42152) and [#42288](BerriAI/litellm#42288) to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42538](BerriAI/litellm#42538) - feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42595](BerriAI/litellm#42595) - chore(release): backport [#42388](BerriAI/litellm#42388) and [#41462](BerriAI/litellm#41462) to stable/1.102.x by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42618](BerriAI/litellm#42618) **Full Changelog**: <BerriAI/litellm@v1.102.0...v1.102.1> </details> --- ### Configuration 📅 **Schedule**: (in timezone Europe/Rome) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.aresu.eu/GiorgioAresu/home-ops/pulls/2200

…02.1) (#267) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/berriai/litellm](https://images.chainguard.dev/directory/image/wolfi-base/overview) ([source](https://github.com/BerriAI/litellm)) | patch | `v1.102.0` → `v1.102.1` | --- ### Release Notes <details> <summary>BerriAI/litellm (ghcr.io/berriai/litellm)</summary> ### [`v1.102.1`](https://github.com/BerriAI/litellm/releases/tag/v1.102.1) [Compare Source](BerriAI/litellm@v1.102.0...v1.102.1) #### Verify Docker Image Signature All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](BerriAI/litellm@0112e53). **Verify using the pinned commit hash (recommended):** A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` **Verify using the release tag (convenience):** Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules: ```bash cosign verify \ --key https://raw.githubusercontent.com/BerriAI/litellm/v1.102.1/cosign.pub \ ghcr.io/berriai/litellm:v1.102.1 ``` Expected output: ``` The following checks were performed on each of these signatures: - The cosign claims were validated - The signatures were verified against the specified public key ``` *** #### What's Changed - fix(anthropic): backport [#42152](BerriAI/litellm#42152) and [#42288](BerriAI/litellm#42288) to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42538](BerriAI/litellm#42538) - feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42595](BerriAI/litellm#42595) - chore(release): backport [#42388](BerriAI/litellm#42388) and [#41462](BerriAI/litellm#41462) to stable/1.102.x by [@devin-ai-integration](https://github.com/devin-ai-integration)\[bot] in [#42618](BerriAI/litellm#42618) **Full Changelog**: <BerriAI/litellm@v1.102.0...v1.102.1> </details> --- ### Configuration 📅 **Schedule**: (in timezone America/New_York) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.greyrock.io/todd/home-ops/pulls/267

feat(typesafe): add TypeSafe Jev passthrough with logging and cost tr…

3b483c8

…acking Backport of #41607 to stable/1.102.x. Cherry-picked from deb9d8a (main). Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot requested a review from a team September 22, 2026 21:36

devin-ai-integration Bot requested a review from mateo-berri as a code owner September 22, 2026 21:36

devin-ai-integration Bot assigned mateo-berri Sep 22, 2026

greptile-apps Bot reviewed Sep 22, 2026

View reviewed changes

Comment thread litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py

cursor Bot reviewed Sep 22, 2026

View reviewed changes

tin-berri and others added 7 commits September 22, 2026 22:49

feat(auto-router): allow opted-in team members to manage their routers

1a3cec8

Prerequisite for #41615 and #41879 on stable/1.102.x. Cherry-picked from 109ca70 (main).

feat(router): add TypeSafe Jev as a complexity router classifier

b5ac613

Backport of #41615 to stable/1.102.x. Cherry-picked from cf42b60 (main).

fix(proxy): forward every method on the typesafe pass-through route

79b72b8

Backport of #41723 to stable/1.102.x. Cherry-picked from 34718f0 (main).

feat(guardrails): add TypeSafe Jev relevance-based compaction guardrail

4b0ffcd

Backport of #41757 to stable/1.102.x. Cherry-picked from 2edda5a (main).

fix(proxy): enforce virtual key budgets for JEV test routing

92a2a2f

Backport of #41879 to stable/1.102.x. Cherry-picked from 1e161f5 (main).

feat(auto-router): add JEV classifier alongside LLM classifier

e20cfbe

Backport of #41886 to stable/1.102.x. Cherry-picked from a83773c (main).

feat(openrouter): price typesafe/jev-1.13 and add an openrouter decis…

d068734

…ions pass-through (#42301) Backport of #42301 to stable/1.102.x. Cherry-picked from 1106b16 (main).

devin-ai-integration Bot requested review from ryan-crabbe-berri and yuneng-berri as code owners September 22, 2026 23:00

devin-ai-integration Bot mentioned this pull request Sep 22, 2026

chore(release): backport #42388 and #41462 to stable/1.102.x #42618

Merged

6 tasks done

fix(ui): adapt the jev dashboard pieces to stable/1.102.x

4bfac88

mateo-berri changed the title ~~feat(typesafe): backport #41607 to stable/1.102.x for v1.102.1~~ feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 Sep 23, 2026

greptile-apps Bot reviewed Sep 23, 2026

View reviewed changes

cursor Bot reviewed Sep 23, 2026

View reviewed changes

mateo-berri approved these changes Sep 23, 2026

View reviewed changes

mateo-berri merged commit 188c9d1 into stable/1.102.x Sep 23, 2026
10 checks passed

mateo-berri deleted the litellm_cherrypick_1_102_x branch September 23, 2026 03:01

This was referenced Sep 23, 2026

chore(backport): add the remaining jev changes to stable/1.100.x #42668

Open

chore(backport): add the remaining jev changes to stable/1.99.x #42669

Open

Conversation

devin-ai-integration Bot commented Sep 22, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (9529383, port 16390)

Chat through the JEV auto router

Messages and Responses through the JEV auto router

Dry-run routing with the JEV classifier

TypeSafe passthrough

OpenRouter decisions passthrough

TypeSafe compaction guardrail

JEV auto router created through the API

TypeSafe guardrail created through the API

Admin UI classifier picker

Existing surfaces (regression sweep)

After (4bfac88, port 16391)

Chat through the JEV auto router

Messages and Responses through the JEV auto router

Dry-run routing with the JEV classifier

TypeSafe passthrough

OpenRouter decisions passthrough

TypeSafe compaction guardrail

JEV auto router created through the API

TypeSafe guardrail created through the API

Admin UI classifier picker

Existing surfaces (regression sweep)

Live PR risk

Type

Caveats (if any)

Medium

Low

Final Attestation

Uh oh!

devin-ai-integration Bot commented Sep 22, 2026

Uh oh!

greptile-apps Bot commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

Uh oh!

mateo-berri commented Sep 22, 2026

Uh oh!

devin-ai-integration Bot commented Sep 22, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

CLAassistant commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mateo-berri commented Sep 23, 2026

Uh oh!

mateo-berri commented Sep 23, 2026

Uh oh!

greptile-apps Bot Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

mateo-berri Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 23, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Sep 23, 2026

Choose a reason for hiding this comment

Member saved JEV probe rejects credentials

devin-ai-integration Bot commented Sep 22, 2026 •

edited by mateo-berri

Loading

Before (`9529383`, port 16390)

After (`4bfac88`, port 16391)

greptile-apps Bot commented Sep 22, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

CLAassistant commented Sep 22, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading