feat(router): add TypeSafe Jev as a complexity router classifier by devin-ai-integration[bot] · Pull Request #41615 · BerriAI/litellm

devin-ai-integration · 2026-09-17T17:08:52Z

TLDR

Problem this solves:

Jev returns a typed choice plus calibrated probabilities, not text
Nothing in the complexity router could use it as the tier classifier

How it solves it:

New classifier_type: jev with jev_classifier_config on the complexity router
Configured tier labels become one Jev choice question; the choice picks the tier
Routing decision records the probabilities, confidence, classifier model, and cost
Failures and timeouts reuse the existing classifier fallback and circuit breaker
The proxy's TYPESAFE_API_KEY is only ever sent to TypeSafe's own host

User Flow

Before: an operator who wants Jev deciding the tier gets no working auto model

They put classifier_type: jev under complexity_router_config on an auto model and start the proxy
The proxy starts, but GET https://litellm-domain/v1/models lists only the three Claude models; auto is missing
A developer sends POST https://litellm-domain/v1/chat/completions with "model": "auto" and a user message
They get 400 You passed in model=auto. There are no healthy deployments for this model

After: the same config classifies through Jev and the spend log explains each pick

They put classifier_type: jev and jev_classifier_config: {model: jev-latest} under complexity_router_config, set TYPESAFE_API_KEY, and start the proxy
GET https://litellm-domain/v1/models now lists auto next to the three Claude models
A developer sends POST https://litellm-domain/v1/chat/completions with "model": "auto" and a user message
They get 200 from the model in the tier Jev chose: a one-word geography question is answered by claude-haiku-4-5, a distributed systems design question by claude-sonnet-5
https://litellm-domain/ui/?page=logs shows each row with Routing cause: jev_classifier, classifier_model: typesafe/jev-1.13.0, per tier probabilities, a confidence value, and the classifier cost
A team member who edits their own auto router through POST https://litellm-domain/auto_router/test_routing cannot set jev_classifier_config.api_key or api_base (422), so the proxy's TypeSafe key never leaves TypeSafe's host

Relevant issues

Follows #41607 (merged), which shipped the typesafe/jev-* registry rows this PR reads for classifier_cost; that merge is on this branch, so the cost populates in the proof below

Affected release

Linear ticket

Resolves LIT-7975

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs boot the proxy from their own worktree with --num_workers 2 --use_v2_migration_resolver, each against its own Postgres database, with real Anthropic models and the real https://api.typesafe.ai (TYPESAFE_API_KEY from the environment). Requests use a virtual key from POST /key/generate. The After proxy also carries auto-db, a second auto router stored in the database by an earlier dry run of the same config, which is why /v1/models lists it too. Shared config:

model_list:
  - model_name: claude-haiku-4-5
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: claude-opus-5
    litellm_params:
      model: anthropic/claude-opus-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: auto
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: claude-haiku-4-5
          MEDIUM: claude-sonnet-5
          COMPLEX: claude-sonnet-5
          REASONING: claude-opus-5
        classifier_type: jev
        jev_classifier_config:
          model: jev-latest
      complexity_router_default_model: claude-sonnet-5
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

The two prompts: What is the capital of France? Answer in one word. and Design a distributed rate limiter that stays correct across three regions with clock skew of up to 200ms. Prove an upper bound on over-admission per window and give the full algorithm with its failure modes., both sent as {"model": "auto", "messages": [{"role": "user", "content": "<prompt>"}], "max_tokens": 120}

Before (`4b368bf`)

Chat completions through `auto`

curl -sS http://127.0.0.1:32590/v1/models -H "Authorization: Bearer $VIRTUAL_KEY" | jq -c "[.data[].id]"
```
["claude-haiku-4-5","claude-sonnet-5","claude-opus-5"]
```

curl -sS -i -X POST http://127.0.0.1:32590/v1/chat/completions -H "Authorization: Bearer $VIRTUAL_KEY" -H "Content-Type: application/json" -d '<prompt 1>'

HTTP/1.1 400 Bad Request
x-litellm-call-id: c6b63a36-620b-4bb3-bfef-8fd1b21c94bc
{"error":{"message":"litellm.BadRequestError: You passed in model=auto. There are no healthy deployments for this model. Received Model Group=auto\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"400"}}

Same for prompt 2

HTTP/1.1 400 Bad Request
x-litellm-call-id: 2433d67f-49d9-483f-bda8-78e6456af342
{"error":{"message":"litellm.BadRequestError: You passed in model=auto. There are no healthy deployments for this model. Received Model Group=auto\nAvailable Model Group Fallbacks=None","type":"invalid_request_error","param":null,"code":"400"}}

http://127.0.0.1:32590/ui/?page=logs shows the two rows as 400s with no routing decision

Team member pointing the Jev call at a host they choose

Not reachable at the merge base: there is no classifier_type: jev to configure. The pre-fix behavior of this PR's own commit 0a66328 is shown under After for contrast

Boot with `jev_classifier_config.api_base` and no `api_key`

Same as the chat case above: the merge base does not know classifier_type: jev, so the auto deployment is skipped at boot and /v1/models never lists it

After (`42f2978`, with `853fd04` on top: a test-only commit that cannot change behavior)

Chat completions through `auto`

curl -sS http://127.0.0.1:29336/v1/models -H "Authorization: Bearer $VIRTUAL_KEY" | jq -c "[.data[].id]"
```
["claude-haiku-4-5","claude-sonnet-5","claude-opus-5","auto","auto-db"]
```

curl -sS -i -X POST http://127.0.0.1:29336/v1/chat/completions -H "Authorization: Bearer $VIRTUAL_KEY" -H "Content-Type: application/json" -d '<prompt 1>'

HTTP/1.1 200 OK
x-litellm-call-id: 67229136-5783-4663-b378-7195333dcd75
x-litellm-model-id: 4b9748d0db2078012c2095dd56003aab650c017c3b70485f276a4d7c1f99cb70
{"model":"auto","content":"Paris","usage":{"prompt_tokens":19,"completion_tokens":4}}

Same for prompt 2

HTTP/1.1 200 OK
x-litellm-call-id: 2d87bdbd-242f-4bb5-8b59-ec8fd9f83e50
x-litellm-model-id: ab2d5bbfcb90d5f456c8a6c4466af1263d1137f29fcf63158edd545dafe1f9dd
{"model":"auto","content":"# Distributed Rate Limiter Across 3 Regions with Bounded Clock Skew\n\n## 1. Problem Setup\n\n","usage":{"prompt_tokens":69,"completion_tokens":5151}}

curl -sS "http://127.0.0.1:29336/spend/logs?request_id=<call id>" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c '.[0] | {model, spend, prompt_tokens, completion_tokens, routing: (.metadata.routing_decision | {tier, cause, routed_model, classifier_model, classifier_confidence, classifier_probabilities, classifier_cost})}'

{"model":"anthropic/claude-haiku-4-5","spend":0.000039,"prompt_tokens":19,"completion_tokens":4,"routing":{"tier":"SIMPLE","cause":"jev_classifier","routed_model":"claude-haiku-4-5","classifier_model":"typesafe/jev-1.13.0","classifier_confidence":1.0,"classifier_probabilities":{"MEDIUM":0.0,"SIMPLE":1.0,"COMPLEX":0.0,"REASONING":0.0},"classifier_cost":0.000017976}}
{"model":"anthropic/claude-sonnet-5","spend":0.05164800000000001,"prompt_tokens":69,"completion_tokens":5151,"routing":{"tier":"COMPLEX","cause":"jev_classifier","routed_model":"claude-sonnet-5","classifier_model":"typesafe/jev-1.13.0","classifier_confidence":0.52,"classifier_probabilities":{"MEDIUM":0.0,"SIMPLE":0.0,"COMPLEX":0.64,"REASONING":0.36},"classifier_cost":0.000019362}}

The same two prompts against the proxy rebooted with the config also carrying api_base: https://api.typesafe.ai and api_key: os.environ/TYPESAFE_API_KEY under jev_classifier_config (the documented way to pin the host): 200 and 200, call ids 627e538c-a441-4a46-be0c-7426cdf726a7 (claude-haiku-4-5, classifier_model: typesafe/jev-1.13.0, classifier_cost: 0.000017976) and ea05a2da-82a7-48ec-bbb2-36d05df4ab54 (claude-sonnet-5, 6204 completion tokens, spend 0.062178)
http://127.0.0.1:29336/ui/?page=logs shows both rows as Success at $0.000039 and $0.051648, and the drawer of the second row shows Routing auto (Auto-Router v2), Tier COMPLEX, Decided by jev_classifier, Routed to claude-sonnet-5, the jev-confidence=0.520000 signal and the per tier probability chips

Team member pointing the Jev call at a host they choose

The member's host is a plain HTTP server that journals each request's method, path, whether the Authorization bearer equals the proxy's TYPESAFE_API_KEY, and the state it received, and answers like Jev with model: attacker-host

At 0a66328 (before the fix), a team member's key calls POST http://127.0.0.1:29336/auto_router/test_routing with "complexity_router_config": {..., "classifier_type": "jev", "jev_classifier_config": {"api_base": "http://127.0.0.1:24058"}}

HTTP 200
{"routed_model":"claude-haiku-4-5", ..., "routing_decision":{..., "cause":"jev_classifier", "classifier_model":"typesafe/attacker-host", ...}}

and the member's host journal gains

{"method": "POST", "path": "/v1/systemone", "authorization_scheme": "Bearer", "authorization_is_the_proxys_TYPESAFE_API_KEY": true, "state_received": "What is the capital of France?"}

At 7f581f6 (after the first fix, before the second), the master key with "jev_classifier_config": {"api_base": "http://127.0.0.1:24058", "api_key": ""} still gets HTTP 200 with classifier_model: typesafe/attacker-host and the journal gains the same line; the member gets HTTP 400 {"detail":"Invalid member auto-router configuration at api_key."}

At the tip, the member request from step 1

HTTP 422
{"detail":[{"type":"value_error","loc":["body","complexity_router_config","jev_classifier_config"],"msg":"Value error, jev_classifier_config.api_base requires jev_classifier_config.api_key: TYPESAFE_API_KEY is only sent to TYPESAFE_API_BASE or https://api.typesafe.ai","input":{"api_base":"http://127.0.0.1:24058"},"ctx":{"error":{}}}]}

At the tip, the blank key request from step 2, with the master key and with the member key alike

HTTP 422
{"detail":[{"type":"value_error","loc":["body","complexity_router_config","jev_classifier_config","api_key"],"msg":"Value error, jev_classifier_config.api_key must be non-empty; omit it to use TYPESAFE_API_KEY","input":"","ctx":{"error":{}}}]}

The member's host journal did not grow during steps 3 and 4, and the same member's supported shape, "jev_classifier_config": {"model": "jev-latest"}, still gets HTTP 200 with cause: jev_classifier and classifier_model: typesafe/jev-1.13.0

Boot with `jev_classifier_config.api_base` and no `api_key`

Boot the tip with the shared config plus api_base: https://api.typesafe.ai under jev_classifier_config and no api_key. The proxy log reads

LiteLLM Router:ERROR: router.py:9057 - Error creating deployment: 1 validation error for ComplexityRouterConfig
jev_classifier_config
  Value error, jev_classifier_config.api_base requires jev_classifier_config.REDACTED is only sent to TYPESAFE_API_BASE or https://api.typesafe.ai [type=value_error, input_value={'model': 'jev-latest', '...ttps://api.typesafe.ai'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.13/v/value_error, ignoring and continuing with other deployments.
INFO:     Application startup complete.

so the proxy comes up without auto, the same way it treats any other invalid deployment (the log redactor blanks the word api_key in the message)

Observations from the run:

Auto-Routers table labels a Jev router "Heuristic": PR causes
Edit dialog wording assumes the heuristic classifier: PR causes
Log drawer shows the raw cause label jev_classifier: PR causes
Tier max_tokens ceiling overrides the caller's max_tokens: 120: PR leaves alone
Rejected Jev config drops auto at boot, startup continues: PR leaves alone

Type

🆕 New Feature

Caveats (if any)

High

Fixed before merge in 7f581f6 and 1e7e5b6, recorded for the decision it took
- A caller-chosen api_base paired with the environment key sent TYPESAFE_API_KEY to that host, and a blank api_key slipped past the first fix
- Chosen: api_base requires an explicit api_key, a blank api_key is rejected, and team members can set neither; operators pin the host with api_base plus api_key: os.environ/TYPESAFE_API_KEY
- Alternatives not taken: rejecting api_base for team members only (the master key path would still leak), or dropping api_base from the config entirely (TYPESAFE_API_BASE would then be the only way to reach a private endpoint)

Low

Admin UI has no Jev option and labels a Jev router "Heuristic" (follow-up issue, UI work is out of this PR's scope)
POST /auto_router/test_routing does not count the Jev call against the caller's budget (a dry run costs about $0.00002 at TypeSafe)
The classifier circuit breaker is per worker, matching the existing LLM classifier
Jev sees text only; images on the newest turn are not forwarded
A rejected jev_classifier_config drops auto at boot and startup continues, the existing behavior for any invalid deployment
Docs land in docs: document the Jev complexity router classifier litellm-docs#1525
classifier_model and the classifier cost use the versioned model TypeSafe reports (jev-1.13.0), not the configured jev-latest alias, the same choice feat(proxy): add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking #41607 makes for the passthrough logs; pricing off the alias was the alternative, and both registry rows carry the same price today

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/b8ce9e101c644969abb9bcc9dfa2c333
Open in Devin Desktop: https://app.devin.ai/desktop/session/b8ce9e101c644969abb9bcc9dfa2c333?variant=devin
Requested by: @mateo-berri

Note

Medium Risk
Changes live request routing and sends prompts to an external classifier using TypeSafe credentials; misconfiguration or classifier outages affect tier picks and spend, though fallbacks and tightened api_base/api_key rules limit credential exfiltration.

Overview
Adds classifier_type: jev to the complexity auto-router so tier selection can go through TypeSafe’s /v1/systemone structured choice API instead of parsing LLM text.

Operators configure jev_classifier_config (model, timeout, instructions, circuit breaker). Tier labels or custom tier_definitions become the Jev choice criteria; the returned tier drives routing. Failures, timeouts, and an open circuit reuse the existing classifier fallback path. Routing logs gain cause: jev_classifier, classifier_probabilities, classifier_confidence, classifier model/cost, and related OpenAPI/schema fields.

Credential rules: the proxy resolves TYPESAFE_API_KEY / TYPESAFE_API_BASE (default https://api.typesafe.ai). api_base without an explicit api_key is rejected so the env key is not sent to arbitrary hosts. Team members editing auto-routers may tune Jev settings but cannot set api_key or api_base via _MemberJevClassifierConfig validation.

^{Reviewed by Cursor Bugbot for commit 853fd04. Bugbot is set up for automated code reviews on this repo. Configure here.}

853fd04 passes /live-pr-risk

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…tor argument Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration · 2026-09-17T17:08:56Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

codspeed · 2026-09-17T17:11:21Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_jev_complexity_classifier (853fd04) with main (02a20fe)}

greptile-apps · 2026-09-17T17:13:02Z

Confidence Score: 5/5

The PR appears safe to merge; no outstanding correctness, security, or repository-rule violations remain.

Findings

Provider Code Misplaced ▶

Summary

Adds TypeSafe Jev as a structured complexity-router classifier.

Introduces Jev configuration, request/response validation, HTTP transport using LiteLLM’s existing handler, pricing lookup, timeout handling, and circuit-breaker integration.
Maps Jev choices to built-in or custom tiers and records model, cost, confidence, and probability metadata in routing decisions.
Restricts member-managed router configurations from overriding TypeSafe credentials or endpoints.
Updates generated dashboard types and adds tests for routing, malformed responses, fallback behavior, permissions, transport, pricing, and circuit breaking.

_{Reviews (6) · Last reviewed commit: "test(router): price the unpriced Jev cos..."}

greptile-apps · 2026-09-17T17:13:08Z

+    async def evaluate(self, request: JevSystemOneRequest, timeout_s: float) -> JevSystemOneResponse:
+        response: Final = await self._http_client.post(  # pyright: ignore[reportUnknownMemberType]  # AsyncHTTPHandler has a dynamic post signature
+            f"{self._api_base}/v1/systemone",
+            json=request.model_dump(mode="json"),
+            headers=MappingProxyType(
+                {
+                    "Authorization": f"Bearer {self._api_key}",
+                    "Content-Type": "application/json",
+                }
+            ),  # pyright: ignore[reportArgumentType]  # HTTP headers are not mutated by AsyncHTTPHandler
+            timeout=timeout_s,
+        )
+        response.raise_for_status()
+        return TypeAdapter(JevSystemOneResponse).validate_python(response.json())


Provider Code Misplaced This TypeSafe transport logic violates the requirement to keep provider-specific code under llms/; move it before merging

Rule Used: What: Avoid writing provider-specific code outside of the llms/ directory. Why: This practice ensures better maintainability and reduces complexity over time. Good: ```python # Handle provider-specific logic within llms/vertex_ai/transformation.py ... (source)

Knowledge Base Used: Provider adapters and capabilities

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Jev is a router classifier adapter, not an LLM provider implementation; its dedicated System One protocol belongs with complexity-router strategy code.

You're right. Jev is being used as a complexity-router classifier, and jev_classifier.py contains the router-specific System One protocol adapter rather than an LLM provider integration. Keeping it under router_strategy/complexity_router/ is appropriate, so this finding should be withdrawn.

Complexity-router classifiers already own classifier-specific transport, while llms adapters serve model completion APIs; moving Jev would misclassify router infrastructure as provider completion code.

codecov · 2026-09-17T17:17:56Z

Codecov Report

❌ Patch coverage is 93.29609% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...er_strategy/complexity_router/complexity_router.py	82.75%	10 Missing ⚠️
...outer_strategy/complexity_router/jev_classifier.py	97.14%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

mateo-berri · 2026-09-17T17:33:21Z

@greptileai

mateo-berri · 2026-09-17T17:33:41Z

@greptileai

mateo-berri · 2026-09-17T17:44:16Z

@greptileai

mateo-berri · 2026-09-17T17:50:15Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

mateo-berri · 2026-09-17T23:09:41Z

@greptileai

mateo-berri · 2026-09-17T23:16:55Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…er-chosen api_base

mateo-berri · 2026-09-17T23:41:03Z

@greptileai

mateo-berri · 2026-09-17T23:48:01Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

…ry never ships

mateo-berri · 2026-09-18T00:18:35Z

@greptileai

mateo-berri · 2026-09-18T00:24:39Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 853fd04. Configure here.}

mateo-berri

LGTM

Partial backport of upstream BerriAI#41607 (price rows and the "evaluation" model mode only; the /typesafe pass-through is not carried), plus the test_models_by_provider skip from BerriAI#41723. The Jev classifier (BerriAI#41615) prices its calls from typesafe/jev-* registry rows, so they must exist even when LITELLM_LOCAL_MODEL_COST_MAP is set. Drop this commit when rebasing onto a release that contains BerriAI#41607. Co-Authored-By: Mateo Wang <277851410+mateo-berri@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Backport of upstream BerriAI#41615 (merge cf42b60) onto v1.101.0 + MongoDB vector store work. Adapted to the 1.101 base: - Jev tier criteria cover the four built-in tiers only (1.101 has no NON_REASONING tier); enable_non_reasoning_tier is not carried. - _with_classifier_forecast only attaches the Jev verdict (1.101 has no capability / llm_v2 forecasts). - auto_router_permissions.py does not exist on 1.101, so the team-member Jev restriction is not carried. JevClassifierConfig still refuses an api_base without its own api_key, so TYPESAFE_API_KEY is never sent to a caller-chosen host. - schema.d.ts is regenerated in a later commit. Drop this commit when rebasing onto a release that contains BerriAI#41615. (cherry picked from commit cf42b60) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Prerequisite for #41615 and #41879 on stable/1.102.x. Cherry-picked from 109ca70 (main).

Backport of #41615 to stable/1.102.x. Cherry-picked from cf42b60 (main).

…pes for stable/1.102.x The committed schema.d.ts and lazy OpenAPI snapshot were taken verbatim from main by the #41615 and #41757 picks, so they described classifier types, routing causes, and a guardrail that this line does not ship and lacked the /auto_router/manage permission. Regenerated under Python 3.12, the interpreter the check-ui-api-types workflow uses.

Prerequisite for #41615 on stable/1.101.x. Cherry-picked from 7015bf3 (main).

Prerequisite for #41615 and #41879 on stable/1.101.x. Cherry-picked from 109ca70 (main).

Backport of #41615 to stable/1.101.x. Cherry-picked from cf42b60 (main).

…pes for stable/1.101.x The #41615 and #41757 picks brought litellm/proxy/_lazy_openapi_snapshot.json and ui/litellm-dashboard/src/lib/http/schema.d.ts verbatim from main. Both files were regenerated under Python 3.12.

mateo-berri and others added 2 commits September 17, 2026 17:05

feat(router): add TypeSafe Jev as a complexity router classifier

9b7fcd0

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

refactor(router): build the Jev client without rebinding the construc…

d7b281c

…tor argument Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot requested a review from a team September 17, 2026 17:08

devin-ai-integration Bot assigned mateo-berri Sep 17, 2026

greptile-apps Bot reviewed Sep 17, 2026

View reviewed changes

fix(router): validate Jev classifier probabilities

0a66328

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

cursor Bot reviewed Sep 17, 2026

View reviewed changes

This was referenced Sep 17, 2026

docs: document the Jev complexity router classifier BerriAI/litellm-docs#1525

Merged

docs: add TypeSafe AI Jev pass-through page BerriAI/litellm-docs#1524

Merged

fix(router): keep the TypeSafe key off caller-chosen Jev endpoints

7f581f6

cursor Bot reviewed Sep 17, 2026

View reviewed changes

Comment thread litellm/router_strategy/complexity_router/config.py

mateo-berri added 2 commits September 17, 2026 16:38

fix(router): reject a blank Jev api_key so it cannot pair with a call…

1e7e5b6

…er-chosen api_base

Merge remote-tracking branch 'origin/main' into HEAD

42f2978

cursor Bot reviewed Sep 17, 2026

View reviewed changes

test(router): price the unpriced Jev cost test off a model the regist…

853fd04

…ry never ships

cursor Bot reviewed Sep 18, 2026

View reviewed changes

mateo-berri approved these changes Sep 18, 2026

View reviewed changes

mateo-berri merged commit cf42b60 into main Sep 18, 2026
91 checks passed

mateo-berri deleted the litellm_jev_complexity_classifier branch September 18, 2026 00:31

wofiporia mentioned this pull request Sep 18, 2026

fix(proxy): return 400 when Jev auto router is missing TypeSafe API key #41739

Open

6 tasks done

This was referenced Sep 18, 2026

fix(proxy): enforce virtual key budgets for JEV test routing #41879

Merged

feat(auto-router): add JEV classifier alongside LLM classifier #41886

Merged

keikeijp mentioned this pull request Sep 20, 2026

Jev (TypeSafe) 導入の経済合理性と実装案 — DeepSeek V4.1 Flash 前提での dtm-agent 評価 keikeijp/keikeijp#1

Open

n24q02m mentioned this pull request Sep 20, 2026

Jev classifier: confidence-gated route-down + fail-expensive option #42139

Open

devin-ai-integration Bot pushed a commit that referenced this pull request Sep 22, 2026

feat(auto-router): allow opted-in team members to manage their routers

1a3cec8

Prerequisite for #41615 and #41879 on stable/1.102.x. Cherry-picked from 109ca70 (main).

devin-ai-integration Bot pushed a commit that referenced this pull request Sep 22, 2026

feat(router): add TypeSafe Jev as a complexity router classifier

b5ac613

Backport of #41615 to stable/1.102.x. Cherry-picked from cf42b60 (main).

devin-ai-integration Bot mentioned this pull request Sep 22, 2026

feat(typesafe): backport the jev change set to stable/1.102.x for v1.102.1 #42595

Merged

14 tasks done

devin-ai-integration Bot pushed a commit that referenced this pull request Sep 23, 2026

fix(proxy): apply team model aliases on the JWT auth path

9250fd5

Prerequisite for #41615 on stable/1.101.x. Cherry-picked from 7015bf3 (main).

devin-ai-integration Bot pushed a commit that referenced this pull request Sep 23, 2026

feat(auto-router): allow opted-in team members to manage their routers

3b97c28

Prerequisite for #41615 and #41879 on stable/1.101.x. Cherry-picked from 109ca70 (main).

devin-ai-integration Bot pushed a commit that referenced this pull request Sep 23, 2026

feat(router): add TypeSafe Jev as a complexity router classifier

2f63249

Backport of #41615 to stable/1.101.x. Cherry-picked from cf42b60 (main).

This was referenced Sep 23, 2026

chore(backport): add the remaining jev changes to stable/1.100.x #42668

Open

chore(backport): add the remaining jev changes to stable/1.99.x #42669

Open

Conversation

devin-ai-integration Bot commented Sep 17, 2026 • edited by mateo-berri Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (4b368bf)

Chat completions through auto

Team member pointing the Jev call at a host they choose

Boot with jev_classifier_config.api_base and no api_key

After (42f2978, with 853fd04 on top: a test-only commit that cannot change behavior)

Chat completions through auto

Team member pointing the Jev call at a host they choose

Boot with jev_classifier_config.api_base and no api_key

Type

Caveats (if any)

High

Low

Final Attestation

Uh oh!

devin-ai-integration Bot commented Sep 17, 2026

🤖 Devin AI Engineer

Uh oh!

codspeed Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

greptile-apps Bot Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

mateo-berri Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

mateo-berri Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

mateo-berri commented Sep 17, 2026

Uh oh!

devin-ai-integration Bot commented Sep 17, 2026 •

edited by mateo-berri

Loading

Before (`4b368bf`)

Chat completions through `auto`

Boot with `jev_classifier_config.api_base` and no `api_key`

After (`42f2978`, with `853fd04` on top: a test-only commit that cannot change behavior)

Chat completions through `auto`

Boot with `jev_classifier_config.api_base` and no `api_key`

codspeed Bot commented Sep 17, 2026 •

edited

Loading

greptile-apps Bot commented Sep 17, 2026 •

edited

Loading

codecov Bot commented Sep 17, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading