tin-berri · Oct 2, 2026 · Sep 28, 2026 · Oct 1, 2026
diff --git a/docs/auto_router/decision_classifiers.md b/docs/auto_router/decision_classifiers.md
@@ -0,0 +1,197 @@
+---
+title: OSS classifiers
+sidebar_label: Jev, Nimble and Laya
+description: Configure the Auto Router's OSS Classifier with hosted Jev or self-hosted Nimble and Laya.
+---
+
+Use Jev, Nimble or Laya to choose which model answers each request. The classifier picks a complexity tier; the Auto Router then calls a model configured for that tier. Clients keep using one router model name through the normal chat completions API.
+
+Jev is available through TypeSafe's hosted API. Nimble and Laya are open models you can run on your own infrastructure. All three use `classifier_type: oss_classifier`, with the provider selected inside `opensource_classifier_config`.
+
+| Classifier | Where it runs | Classifier model | `opensource_classifier_config.provider` |
+| --- | --- | --- | --- |
+| [Jev](https://docs.typesafe.ai/) | TypeSafe's hosted API | `jev-latest` | `jev` |
+| [Bespoke Nimble](https://github.com/bespokelabsai/nimble) | Self-hosted, including Ollama | `nimble` on Ollama; `nimble-latest` on Bespoke's System One server | `jev` with a custom endpoint |
+| [Laya](https://github.com/NandhaKishorM/laya) | Self-hosted | `english`, `multilingual` or `typed-decisions` | `laya` |
+
+All three use the System One decision protocol. LiteLLM sends a `choice` question describing your tiers to `POST /v1/systemone`. Set `api_base` to the server's base URL without `/v1/systemone`; an endpoint that only exposes `/v1/evaluate` or chat completions is not sufficient.
+
+:::info Availability
+
+The configuration names in this guide and Laya support require a gateway build containing [backend #43626](https://github.com/BerriAI/litellm/pull/43626). The **OSS Classifier** dashboard selector also requires [UI #43768](https://github.com/BerriAI/litellm/pull/43768). These changes are pending; existing releases can reject `oss_classifier`, `opensource_classifier_config` and `provider: laya`.
+
+For Jev or Nimble on a released build, keep `classifier_type: jev` and `jev_classifier_config` as shown in the [Jev setup guide](/docs/auto_router/setup#jev-classifier-typesafe-ai). The new backend continues to accept those names; see [migration](#migrate-an-existing-jev-or-nimble-router).
+
+:::
+
+## Configure the router
+
+Start with this complete configuration, then choose the classifier connection below. `small-solver` and `large-solver` are public deployment names; replace their underlying models with models your gateway can access:
+
+```yaml title="config.yaml"
+model_list:
+  - model_name: small-solver
+    litellm_params:
+      model: openai/{{openai_small}}
+      api_key: os.environ/OPENAI_API_KEY
+  - model_name: large-solver
+    litellm_params:
+      model: openai/{{openai_large}}
+      api_key: os.environ/OPENAI_API_KEY
+  - model_name: decision-router
+    litellm_params:
+      model: auto_router/complexity_router
+      complexity_router_default_model: large-solver
+      complexity_router_config:
+        classifier_type: oss_classifier
+        opensource_classifier_config:
+          provider: jev
+          model: jev-latest
+          timeout_ms: 3000
+        tiers:
+          SIMPLE: small-solver
+          MEDIUM: small-solver
+          COMPLEX: large-solver
+          REASONING: large-solver
+        classifier_fallback: default_model
+        classifier_context_window_size: 0
+```
+
+This example routes classifier failures to `large-solver` and sends no prior conversation turns to the classifier. The current request and selected system text are still sent. The default fallback, when omitted, is `heuristic`; the default history window is three prior user turns within an 8,000-character prior-turn budget.
+
+### Jev: hosted TypeSafe API
+
+Set `TYPESAFE_API_KEY` in the gateway's environment using your secret manager. Leave the example's `opensource_classifier_config` unchanged. `provider: jev` still uses the TypeSafe transport and environment variables. `TYPESAFE_API_BASE` is optional and defaults to `https://api.typesafe.ai`.
+
+For a router-specific endpoint, supply both `api_base` and its matching `api_key` inside `opensource_classifier_config`. An explicit endpoint does not inherit the TypeSafe environment key. The model, timeout, instructions and circuit-breaker fields follow the [Jev reference](/docs/proxy/auto_routing#jev-classifier), whose examples retain the names supported by released builds.
+
+### Nimble: self-hosted with Ollama
+
+Install [Ollama 0.35 or later](https://ollama.com/library/nimble) and start its server. If the Ollama app is not already serving requests, run `ollama serve` in a separate terminal. Download the decision model:
+
+```bash
+ollama pull nimble
+```
+
+Replace the example's `opensource_classifier_config` with this block:
+
+```yaml
+opensource_classifier_config:
+  provider: jev
+  model: nimble
+  api_base: http://localhost:11434
+  api_key: ollama
+  timeout_ms: 30000
+```
+
+Local Ollama ignores the API key. `ollama` is a non-secret placeholder because LiteLLM's Jev-compatible client requires a nonempty key when an endpoint is supplied. If your endpoint has authentication, use that endpoint's real credential instead. There is no `provider: nimble` setting.
+
+The longer timeout allows more time for local inference; warm the model before measuring latency. For Bespoke's own System One server, use its base URL and `model: nimble-latest` instead. See the upstream [serving guide](https://github.com/bespokelabsai/nimble/blob/main/docs/MODAL_SERVING.md) and [model preparation guide](https://github.com/bespokelabsai/nimble#quickstart).
+
+### Laya: self-hosted HTTP server
+
+Install Laya in a separate environment from LiteLLM and start the English checkpoint:
+
+```bash
+python -m venv .venv-laya
+source .venv-laya/bin/activate
+python -m pip install 'laya[serve]==0.3.21'
+LAYA_HOST=127.0.0.1 LAYA_PORT=8000 LAYA_MODELS=english laya-serve
+```
+
+In the gateway process, set its reachable server URL:
+
+```bash
+export LAYA_API_BASE="http://127.0.0.1:8000"
+```
+
+Replace the example's `opensource_classifier_config` with this block:
+
+```yaml
+opensource_classifier_config:
+  provider: laya
+  model: english
+  timeout_ms: 15000
+```
+
+Choose `english`, `multilingual` or `typed-decisions` explicitly. When `api_base` is omitted, the gateway uses `LAYA_API_BASE` and optional `LAYA_API_KEY`. To require authentication, configure the same key on the Laya server and gateway.
+
+An administrator can instead set `api_base` and optional `api_key` on this router. An explicit Laya endpoint without a key connects without authentication and does not inherit the environment key. See the [Laya HTTP guide](https://github.com/NandhaKishorM/laya/blob/main/docs/http-api.md) for server options.
+
+For either self-hosted server, `localhost` refers to the gateway's own host or container. Use a hostname reachable from the gateway when the classifier runs elsewhere.
+
+## Test routing and send a request
+
+Start LiteLLM from its own environment after configuring the completion-provider credentials and your chosen classifier:
+
+```bash
+litellm --config config.yaml
+```
+
+In **Models + Endpoints → Auto Router**, open the create or edit form and use **Test Routing** to inspect the chosen tier and model without calling the completion model. A successful fallback does not prove the classifier worked: inspect the routing cause and classifier model. A decision-model result uses `cause: jev_classifier`; Laya reports a classifier model such as `laya/english`.
+
+Use a LiteLLM virtual key with access to `decision-router` to make a completion:
+
+```bash
+curl http://localhost:4000/v1/chat/completions \
+  -H "Authorization: Bearer $LITELLM_API_KEY" \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "decision-router",
+    "messages": [{"role": "user", "content": "Explain what an HTTP status code means"}]
+  }'
+```
+
+In dashboards containing [UI #43768](https://github.com/BerriAI/litellm/pull/43768), select **OSS Classifier** in the auto-router form, then **Jev** or **Laya**. Nimble uses the **Jev** provider with a custom model and endpoint; YAML is the simplest setup. Older dashboards label the classifier **JEV Classifier**.
+
+Administrators can configure connection overrides. Team members cannot submit classifier endpoint or credential overrides through the management API. Saved connection fields are hidden on edit; leaving them untouched preserves them. Use the explicit reset controls to clear a saved key or return to the gateway connection.
+
+## Call Laya's native decision API
+
+With `LAYA_API_BASE` configured as above, call `POST /laya/v1/systemone` using a LiteLLM virtual key with access to `laya/english`. The request names the bare checkpoint; LiteLLM uses `laya/english` for model permissions and usage logs.
+
+```bash
+curl http://localhost:4000/laya/v1/systemone \
+  -H "Authorization: Bearer $LITELLM_API_KEY" \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "model": "english",
+    "state": "My invoice has two identical charges",
+    "questions": {
+      "department": {
+        "type": "choice",
+        "instructions": "Choose the department that should help",
+        "criteria": {
+          "billing": "Invoices, payments, and refunds",
+          "technical": "Bugs and connectivity problems"
+        }
+      }
+    }
+  }'
+```
+
+Every native request must explicitly choose `english`, `multilingual` or `typed-decisions`, with permission for the corresponding `laya/<checkpoint>` model. Unknown names and automatic selection are rejected. The response preserves Laya's `answers`, `usage` and `routing` fields. The native Laya route supports System One requests only; `/laya/v1/evaluate` and Laya chat completions are not exposed.
+
+## Evaluate quality and cost
+
+Compare tier choices on representative prompts before changing production routing. Probabilities describe the supplied choices; confidence scores from different model families are not interchangeable accuracy estimates. Measure downstream answer quality, classifier latency, fallback frequency and total cost using the [evaluation guide](/docs/auto_router/evaluate).
+
+Jev calls can incur TypeSafe charges. Self-hosting Nimble or Laya has compute costs even without a hosted inference fee. Laya's built-in catalog rates are zero; custom Nimble model names may have no registered rate, so missing classifier cost does not establish zero cost. Jev and Nimble retain `typesafe/<model>` classifier log naming; Laya uses `laya/<checkpoint>`. The OSS classifier name does not change the `cause: jev_classifier` value in routing results.
+
+If the classifier falls back, check the endpoint, checkpoint name, credentials, model warm-up and timeout. A timeout can also open the classifier circuit breaker, which defaults to a 30-second recovery interval. Verify that the server accepts `/v1/systemone`, rather than adding that path to `api_base`.
+
+## Migrate an existing Jev or Nimble router
+
+After upgrading to a build containing [backend #43626](https://github.com/BerriAI/litellm/pull/43626), use `classifier_type: oss_classifier` and rename `jev_classifier_config` to `opensource_classifier_config`. Set `provider: jev` and keep the existing model, endpoint, credential and classifier options. For example, the hosted Jev configuration becomes:
+
+```yaml
+classifier_type: oss_classifier
+opensource_classifier_config:
+  provider: jev
+  model: jev-latest
+  timeout_ms: 3000
+```
+
+The new backend still accepts `classifier_type: jev`, `jev_classifier_config` and the legacy `provider: typesafe` value for existing YAML and saved routers. Supply only one classifier configuration block. Model-management creates and updates that include the classifier configuration write the canonical names; an update that leaves the configuration untouched preserves the saved value.
+
+Existing Jev and Nimble connections continue to use `TYPESAFE_API_KEY` and `TYPESAFE_API_BASE` when no explicit connection is supplied. The transport, `typesafe/<model>` accounting labels and `cause: jev_classifier` remain unchanged.
diff --git a/docs/auto_router/index.md b/docs/auto_router/index.md
@@ -82,7 +82,7 @@ curl -X POST http://localhost:4000/v1/chat/completions \
 
 ## Explore
 
-To use TypeSafe System One for classification, choose `classifier_type: jev` and configure `jev_classifier_config`. Start with [JEV setup](/docs/auto_router/setup#jev-classifier-typesafe-ai) and review its [context, fallback and accounting](/docs/proxy/auto_routing#jev-classifier)
+Use the **OSS Classifier** with hosted Jev or self-hosted Nimble and Laya. The [OSS classifier guide](/docs/auto_router/decision_classifiers) covers setup, required gateway and dashboard builds, and migration from the existing Jev configuration. See the [classifier reference](/docs/proxy/auto_routing#jev-classifier) for context, fallback and accounting.
 
 <NavigationCards
 columns={3}

diff --git a/docs/auto_router/setup.md b/docs/auto_router/setup.md
@@ -79,6 +79,8 @@ model_list:
 
 ## JEV classifier (TypeSafe AI)
 
+This section uses the Jev configuration and dashboard labels supported by released builds. The new **OSS Classifier** configuration requires [backend #43626](https://github.com/BerriAI/litellm/pull/43626), and its dashboard requires [UI #43768](https://github.com/BerriAI/litellm/pull/43768). See [OSS classifiers](/docs/auto_router/decision_classifiers) for hosted Jev, self-hosted Nimble and Laya, and the [migration from the existing names](/docs/auto_router/decision_classifiers#migrate-an-existing-jev-or-nimble-router). The new backend continues to accept the configuration below.
+
 `classifier_type: jev` uses TypeSafe System One Choice evaluation to select a tier inside the existing Auto Router. LiteLLM sends the classifier input to `POST /v1/systemone` as `state`, with one `questions.tier` question whose criteria describe the configured tiers. The chosen tier's model serves the completion
 
 ### Set the server key

diff --git a/docs/pass_through/typesafe.md b/docs/pass_through/typesafe.md
@@ -15,6 +15,8 @@ LiteLLM adds the TypeSafe API key from the proxy environment, so clients only ne
 
 To let JEV pick the model for a completion, configure the [JEV Auto Router](/docs/auto_router/setup#jev-classifier-typesafe-ai) with `classifier_type: jev` and `jev_classifier_config`. It uses one System One Choice question for the configured tiers, then dispatches to the selected completion model. See [routing context, fallback and accounting](/docs/proxy/auto_routing#jev-classifier) and the [measured classifier comparison](/blog/jev-auto-router-benchmark)
 
+The [OSS classifier guide](/docs/auto_router/decision_classifiers) documents the canonical `classifier_type: oss_classifier` and `opensource_classifier_config.provider: jev` names, which require a gateway build containing [backend #43626](https://github.com/BerriAI/litellm/pull/43626). Both configurations use the TypeSafe transport; the new backend continues to accept the existing Jev names.
+
 ## Quick Start
 
 1. Set the TypeSafe API key in the proxy environment

diff --git a/docs/proxy/auto_routing.md b/docs/proxy/auto_routing.md
@@ -203,6 +203,8 @@ classifier_llm_config:
 
 ### JEV classifier
 
+This reference retains the field names supported by released builds. A build containing [backend #43626](https://github.com/BerriAI/litellm/pull/43626) also accepts `classifier_type: oss_classifier` with `opensource_classifier_config.provider: jev`; the settings below apply inside `opensource_classifier_config`. The existing names remain accepted. See the [OSS classifier guide](/docs/auto_router/decision_classifiers) for migration and the Jev, Nimble and Laya provider configurations.
+
 Set `classifier_type: jev` with `jev_classifier_config` to use [TypeSafe System One](/docs/pass_through/typesafe). It sends one `questions.tier` Choice question to `POST /v1/systemone`, with the classification input in `state` and tier descriptions in `criteria`. The returned choice selects the existing tier pool. See [setup and dashboard instructions](/docs/auto_router/setup#jev-classifier-typesafe-ai)
 
 ```yaml

diff --git a/docs/proxy/config_settings.md b/docs/proxy/config_settings.md
@@ -799,6 +799,8 @@ router_settings:
 | JINA_API_KEY | Fallback for `JINA_AI_API_KEY`
 | LANGFLOW_API_BASE | Base URL for Langflow. Default is http://localhost:7860
 | LANGFLOW_API_KEY | API key for Langflow
+| LAYA_API_BASE | HTTP(S) base URL of a self-hosted Laya server, without `/v1/systemone`, for native gateway decisions and [OSS classifiers](/docs/auto_router/decision_classifiers) with `opensource_classifier_config.provider: laya` that omit `api_base`. Required when no classifier endpoint is supplied. Requires a build containing [LiteLLM #43626](https://github.com/BerriAI/litellm/pull/43626) |
+| LAYA_API_KEY | Optional bearer key paired with `LAYA_API_BASE`. A Laya classifier with an explicit `opensource_classifier_config.api_base` uses only its explicit `api_key`; omitting that key connects without authentication |
 | LEMONADE_API_KEY | API key for Lemonade
 | LINKUP_API_BASE | Base URL for the Linkup search provider
 | LLAMAFILE_API_KEY | API key for llamafile. llamafile does not require one, so a placeholder is used when this is unset

diff --git a/sidebars.js b/sidebars.js
@@ -1886,6 +1886,7 @@ const autoRouterSidebar = {
   autoRouterSidebar: [
     { type: "doc", id: "auto_router/index", label: "Overview", className: "autorouter-nav-item" },
     { type: "doc", id: "auto_router/setup", className: "autorouter-nav-item" },
+    { type: "doc", id: "auto_router/decision_classifiers", className: "autorouter-nav-item" },
     { type: "doc", id: "auto_router/user_setup", className: "autorouter-nav-item" },
     { type: "doc", id: "auto_router/recommended_configurations", className: "autorouter-nav-item" },
     { type: "doc", id: "auto_router/benchmarks", className: "autorouter-nav-item" },
-Original file line number
+Diff line change
@@ Expand Up / @@ -203,6 +203,8 @@ classifier_llm_config: @@
     ### JEV classifier
+    This reference retains the field names supported by released builds. A build containing [backend #43626](https://github.com/BerriAI/litellm/pull/43626) also accepts `classifier_type: oss_classifier` with `opensource_classifier_config.provider: jev`; the settings below apply inside `opensource_classifier_config`. The existing names remain accepted. See the [OSS classifier guide](/docs/auto_router/decision_classifiers) for migration and the Jev, Nimble and Laya provider configurations.
     Set `classifier_type: jev` with `jev_classifier_config` to use [TypeSafe System One](/docs/pass_through/typesafe). It sends one `questions.tier` Choice question to `POST /v1/systemone`, with the classification input in `state` and tier descriptions in `criteria`. The returned choice selects the existing tier pool. See [setup and dashboard instructions](/docs/auto_router/setup#jev-classifier-typesafe-ai)
     ```yaml
@@ Expand Down @@