feat(ui): working Test Connection for the complexity auto router

[akapur99]

Relevant issues

Fixes #31590

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Captured at commit ddc13b3

Context: when we consolidated the auto-router UI in #32859 the Test Connection button was removed, because the shared prepareModelAddRequest helper returns an empty array for an auto router (it has no model_mappings) and the caller crashed destructuring result[0].litellmParamsObj. That is the crash reported in #31590 and the open PR #31794.

Two approaches were considered. #31794, and my first cut, routed the test through /health/test_connection. That does not actually work for an auto router: the endpoint merges {...configParams, ...requestParams}, so passing the public model_group name as the request model overrides the resolved provider model and every tier fails with "LLM Provider NOT provided". The frontend only has the public group name, not the underlying litellm_params, so it cannot build the request that endpoint needs.

So this tests each model group the way production actually routes it: a minimal request to /v1/chat/completions (or /v1/embeddings for the embedding model) by public group name, through the shared apiClient. The router resolves the group, credentials, and provider itself, so a green row means the tier is genuinely reachable and a red row shows the real provider error.

Verified live against real APIs at commit ddc13b3:

# embedding tier -> 200 (real Voyage call)
curl -s -X POST http://localhost:4000/v1/embeddings \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"voyage-4-large","input":"test from litellm"}'
# -> {"data":[{"embedding":[... 1024 dims ...]}], ...}

# chat tier -> real provider error surfaces per-row (here an expired key)
curl -s -X POST http://localhost:4000/v1/chat/completions \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"anthropic-sonnet-4-5","messages":[{"role":"user","content":"test from litellm"}],"max_tokens":1}'
# -> {"error":{"message":"litellm.AuthenticationError: ..."}}

UI flow (screenshots to follow): http://localhost:3000 -> Models & Endpoints -> Add Model -> Auto Router tab -> keep "Auto-Router v2" selected, set a name, assign a model group to each complexity tier, optionally enable Semantic Keyword Matching with an embedding model, then click "Test Connection". The result modal shows one row per distinct tier model group (tiers sharing a group collapse into one) plus an Embedding row, each green on success or red with the provider error, updating progressively as each probe returns.

Type

🆕 New Feature
🐛 Bug Fix

Changes

New pure helper build_auto_router_test_targets.ts turns the configured tiers plus the optional embedding model into a deduped list of probe targets, so tiers sharing a model group cost one call instead of several; a satisfies Record<keyof ComplexityTiers, null> guard makes adding a tier without listing it a compile error. New testModelGroupConnection in networking.tsx sends the minimal chat/embedding request through the shared apiClient. New auto_router_connection_test.tsx runs one probe per target and renders per-target status, updating each row as it settles. add_auto_router_tab.tsx gets the Test Connection button (recommended router only) and a results modal that remounts per run via a key. Backend is unchanged.

Tests cover the target builder (dedup, empty-tier dropping, embedding gating) and the component (one request per target with the right mode, success and provider-error rendering, litellm-prefix stripping). The component never calls prepareModelAddRequest, which is the guard against the #31590 regression.

Supersedes #31794

[greptile-apps]

Greptile Summary

This PR restores the Test Connection button for the auto-router UI by routing minimal chat/embedding requests through the proxy by public model-group name, rather than trying to resolve litellm_params before the router config is saved (the root cause of #31590).

• Adds build_auto_router_test_targets.ts — deduplicates complexity tiers into a typed probe list, with a satisfies Record<keyof ComplexityTiers, null> guard to catch missing tiers at compile time.
• Adds auto_router_connection_test.tsx — fires probes in parallel, updates each row's state individually as results arrive, and guards against stale state updates via a cancellation flag on unmount.
• Adds testModelGroupConnection and buildModelGroupTestRequest to networking.tsx, using the existing apiClient so auth and base-URL resolution are handled consistently; the parent remounts the component via an incrementing key for each fresh test run.

Confidence Score: 5/5

Safe to merge — frontend-only change that adds a new Test Connection capability without touching any backend or existing request paths.

All changed code is isolated to the auto-router UI tab and the networking layer additions. The implementation correctly reuses the existing apiClient for auth and base-URL handling, cleanly handles cancellation on unmount, and has good unit-test coverage for both the target builder and the component. The only finding is a misplaced JSDoc comment in networking.tsx with no runtime impact.

No files require special attention.

Important Files Changed

Filename	Overview
ui/litellm-dashboard/src/components/add_model/build_auto_router_test_targets.ts	New helper that deduplicates complexity tiers into probe targets; the satisfies Record<keyof ComplexityTiers, null> guard enforces exhaustiveness at compile time
ui/litellm-dashboard/src/components/add_model/auto_router_connection_test.tsx	New component that fires one probe per target in parallel with per-row progressive state updates and a cancellation guard on unmount
ui/litellm-dashboard/src/components/add_model/add_auto_router_tab.tsx	Adds Test Connection button (recommended router only), builds targets from current form state, and remounts the test component via incrementing key for each fresh run
ui/litellm-dashboard/src/components/networking.tsx	Adds buildModelGroupTestRequest and testModelGroupConnection; two JSDoc blocks appear in reversed order, leaving testModelGroupConnection without a JSDoc and the first block orphaned
ui/litellm-dashboard/src/components/add_model/build_auto_router_test_targets.test.ts	Comprehensive unit tests for target builder covering dedup, empty-tier dropping, and embedding gating
ui/litellm-dashboard/src/components/add_model/auto_router_connection_test.test.tsx	Component tests cover per-target request dispatch, success/error rendering, and litellm-prefix stripping
ui/litellm-dashboard/src/components/networking.test.ts	New tests verify buildModelGroupTestRequest output shape for both chat and embedding modes, and confirm no max_tokens is set

_{Reviews (2): Last reviewed commit: "fix(ui): drop max_tokens from the auto-r..." | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_auto_router_test_connection (a80f856) with litellm_internal_staging (e7f4144)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (5e1928c) during the generation of this report, so e7f4144 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[akapur99]

Both P2 notes from the last review are already handled in ddc13b3, which also reworks the probe to route through /v1/chat/completions and /v1/embeddings (the /health/test_connection path could not resolve a public model_group name). Rows now update progressively as each probe settles, and TIER_ORDER is derived through a satisfies Record<keyof ComplexityTiers, null> guard so a new tier that is not listed is a compile error

@greptileai

[greptile-apps]

Both P2s are confirmed addressed:

Progressive updates (thread 1): The current code already achieves per-row updates. Each async callback inside Promise.all calls setResults((prev) => prev.map(...)) individually as its own probe settles — not after the whole batch. The suggestion in my original comment proposed the same pattern that's already there.

TIER_ORDER exhaustiveness (thread 2): Marked outdated by GitHub, and the satisfies Record<keyof ComplexityTiers, null> guard is equivalent to the typed map I suggested — adding a new key to ComplexityTiers without updating the object literal will produce a compile error. That's exactly what was asked for.

The implementation looks good. LGTM.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

LGTM; thanks!