Skip to content

feat(ui): show in the log drawer and session sidebar when an auto-router served a request - #34434

Merged
tin-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_lit4748_autorouter_logs
Jul 24, 2026
Merged

feat(ui): show in the log drawer and session sidebar when an auto-router served a request#34434
tin-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_lit4748_autorouter_logs

Conversation

@tin-berri

@tin-berri tin-berri commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • The logs already show which model served a request, but nothing shows that an auto-router picked it
  • The requested name is already on every spend-log row as model_group, and both UI endpoints already return it; the dashboard type dropped the field

How it solves it:

  • The request drawer header names the router in a badge, and the session sidebar swaps that entry's leading icon
  • Decides "is this an auto-router" from the deployment list rather than from model_group != model, which is true for ordinary aliases too

Relevant issues

  • Makes it visible in the request drawer and the session view that an auto-router chose the model
  • Reads the auto-router set once per logs tree and shares it through context, so the two surfaces cannot disagree
  • No backend, schema or migration change; the data was already on the wire

Linear ticket

Resolves LIT-4748

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Ran a local proxy against real Anthropic with one auto-router (smart-router, a complexity router over claude-haiku / claude-sonnet / claude-opus) plus two plain aliases, then read back what the UI endpoints serve

for M in smart-router claude-haiku claude-sonnet; do
  curl -s http://localhost:4000/v1/chat/completions \
    -H "Authorization: Bearer sk-lit4748" -H "Content-Type: application/json" \
    -d "{\"model\":\"$M\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi in 3 words\"}],\"max_tokens\":20,\"litellm_session_id\":\"lit4748-demo\"}"
done

curl -s "http://localhost:4000/spend/logs/ui?page=1&page_size=5&start_date=2020-01-01%2000:00:00&end_date=2030-01-01%2000:00:00" \
  -H "Authorization: Bearer sk-lit4748"

Both /spend/logs/ui and /spend/logs/session/ui return model_group on every row

model_group='claude-sonnet' model='anthropic/claude-sonnet-4-5'
model_group='claude-haiku'  model='anthropic/claude-haiku-4-5'
model_group='smart-router'  model='anthropic/claude-haiku-4-5'

This is also why the indication is not driven by comparing the two columns. All three rows have model_group != model, because that is what a model_name alias is; only the third row went through an auto-router. The auto-routed row is otherwise indistinguishable server side, with the same model_id as the direct claude-haiku row and an identical metadata key set, so the requested name is matched against the deployments whose litellm_params.model starts with auto_router/

$ curl -s http://localhost:4000/v2/model/info -H "Authorization: Bearer <key>"
model_name='claude-haiku'  litellm_params.model='anthropic/claude-haiku-4-5'
model_name='claude-sonnet' litellm_params.model='anthropic/claude-sonnet-4-5'
model_name='claude-opus'   litellm_params.model='anthropic/claude-opus-4-8'
model_name='smart-router'  litellm_params.model='auto_router/complexity_router'

Verified that a non-admin virtual key gets the same litellm_params.model values from that endpoint, so the indication renders for non-admins too

Type

🆕 New Feature

Changes

  • LogEntry carries model_group, and useAutoRouterModelGroups resolves the set of auto-router model groups from every page of /v2/model/info, keyed on the auto_router/ prefix so complexity, adaptive and quality routers are all covered
  • New AutoRouterTag and AutoRouterModelGroupsProvider in shared/table_cells, mounted once in the logs panel so both surfaces share one lookup
  • Drawer header shows a badge naming the router next to the provider; the session sidebar entry swaps its leading icon
  • TraceEventRow's icon choice moves into a small TraceEventIcon with early returns, which drops two grandfathered no-nested-ternary suppressions
  • Nothing renders when the model list has not loaded, when the request failed, or outside the provider, so a row is never mislabelled

Things a reviewer will ask about

The indication is a lookup rather than a string comparison because model_group != model holds for every plain alias and every wildcard deployment, so a mismatch-based badge would fire on nearly every row and single out nothing. AutoRouterTag reads its set from context with an empty-set default instead of calling the query hook itself, so any consumer outside a provider keeps rendering without a QueryClient

Left alone deliberately: model_info_view.tsx and edit_auto_router_modal.tsx hand-roll a narrower auto_router/complexity_router check. That asks a different question, namely whether the complexity-router edit UI applies, and widening it to the general prefix would wrongly show that UI for adaptive and quality routers

QA runbook

  1. Start a proxy with an auto-router and at least one plain alias, for example a smart-router model whose litellm_params.model is auto_router/complexity_router alongside a claude-haiku model whose litellm_params.model is anthropic/claude-haiku-4-5
  2. Send three requests so there is one routed row and two direct rows, passing the same litellm_session_id on all three so they group into one session
for M in smart-router claude-haiku claude-sonnet; do
  curl -s http://localhost:4000/v1/chat/completions \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
    -d "{\"model\":\"$M\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi in 3 words\"}],\"max_tokens\":20,\"litellm_session_id\":\"lit4748-demo\"}"
done
  1. Go to http://localhost:4000/ui/?page=logs and click the smart-router row. The drawer header shows the resolved model, the provider, then a badge naming smart-router
  2. Click a claude-haiku or claude-sonnet row. The header shows no badge, exactly as before this PR
  3. Click the session id to open the session view. In the left trace list the auto-routed entry carries the router icon while the two direct entries keep the default icon
  4. Remove the auto-router from the config and reload. Every row and every session entry renders exactly as before this PR

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

Low Risk
Dashboard-only, read-only UI over existing log and model-info APIs; mislabeling is avoided by not rendering until the router set is known.

Overview
Request logs now surface auto-router routing in the drawer header and session trace sidebar, using model_group from spend-log rows and a shared deployment lookup instead of comparing requested vs resolved model names.

LogEntry gains optional model_group. useAutoRouterModelGroups loads all pages of /v2/model/info, treats deployments whose litellm_params.model starts with auto_router/ as routers, and exposes their public model_name set. AutoRouterModelGroupsProvider mounts once on the request logs panel so drawer header and sidebar share one cached lookup.

The drawer header shows an AutoRouterTag badge naming the router when model_group matches that set. Session sidebar rows use the waypoints icon for auto-routed entries (via TraceEventIcon). Tags and icons stay hidden while the lookup is loading, on failure, or outside the provider so rows are not mislabeled.

Reviewed by Cursor Bugbot for commit 42aba4f. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds auto-router indicators to request logs and session details.

  • Fetches every page of model deployments and derives auto-router model groups from the auto_router/ prefix
  • Shares the derived groups through a logs-level React context
  • Displays the router name in the request drawer and a router icon in session entries
  • Extends the spend-log UI type with model_group and adds focused tests

Confidence Score: 5/5

The PR appears safe to merge

No blocking failures remain

Important Files Changed

Filename Overview
ui/litellm-dashboard/src/app/(dashboard)/hooks/models/useModels.ts Adds a paginated query that derives the complete set of configured auto-router model groups
ui/litellm-dashboard/src/components/shared/table_cells/AutoRouterTag.tsx Adds shared context, tag, icon, and lookup helpers for auto-router indicators
ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/LogDetailsDrawer.tsx Uses the shared lookup to select the auto-router icon for session trace entries
ui/litellm-dashboard/src/components/view_logs/LogDetailsDrawer/DrawerHeader.tsx Shows the requested auto-router model group beside model and provider details
ui/litellm-dashboard/src/components/view_logs/RequestLogsPanel.tsx Mounts one auto-router model-group provider around the logs surfaces

Reviews (3): Last reviewed commit: "feat(ui): show in the log drawer and ses..." | Re-trigger Greptile

Comment thread ui/litellm-dashboard/src/app/(dashboard)/hooks/models/useModels.ts
Comment thread ui/litellm-dashboard/src/app/(dashboard)/hooks/models/useModels.ts Outdated
@codspeed-hq

codspeed-hq Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_lit4748_autorouter_logs (e29c2d8) with litellm_internal_staging (43e7b96)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (e906a7e) during the generation of this report, so 43e7b96 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@tin-berri
tin-berri force-pushed the litellm_lit4748_autorouter_logs branch from 950862e to e29c2d8 Compare July 23, 2026 23:37
@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

…ter served a request

The dashboard already receives the requested model name as model_group on
every spend-log row, but LogEntry dropped the field, so nothing distinguished
an auto-routed request from a direct one.

Surface it precisely rather than by comparing requested against resolved:
model_group differs from model for plain aliases and wildcard deployments
too, so a bare mismatch tags almost every row and identifies nothing. The
indication is driven instead by which deployments are auto-routers, resolved
from every page of /v2/model/info and shared through context.

The request drawer header names the router in a badge next to the provider;
the session sidebar swaps the entry's leading icon. Rows that no auto-router
served render exactly as before.
@tin-berri
tin-berri force-pushed the litellm_lit4748_autorouter_logs branch from e29c2d8 to 42aba4f Compare July 24, 2026 03:43
@tin-berri tin-berri changed the title feat(ui): show in Logs and Session views when an auto-router served a request feat(ui): show in the log drawer and session sidebar when an auto-router served a request Jul 24, 2026
@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Runtime test of the latest commit (42aba4f32a)

Ran the branch locally against real Anthropic APIs with a smart-router deployment (auto_router/complexity_router, tiers -> anthropic haiku/sonnet/opus), sent fresh chat requests through it, then inspected the Logs page and a log detail drawer in the Admin UI dev server

This commit moves the auto-router indicator out of the Logs table and into the log detail drawer: a Waypoints badge with the router group name in the header, and a Waypoints icon (replacing Sparkles) on the auto-routed event in the trace sidebar

Drawer header shows the Waypoints "smart-router" badge; Request Details has no badge

Drawer header badge

Logs table no longer shows the old pill under the model

Logs table Model column

Trace sidebar uses the Waypoints icon for the auto-routed event

Trace sidebar Waypoints icon

Assertions, all passed

  • The Logs table shows only the resolved model with no auto-router pill
  • The drawer header shows a Waypoints badge reading smart-router, and Request Details shows the model with no badge
  • The auto-routed event in the trace sidebar uses the Waypoints icon instead of Sparkles

Tested by Devin: https://app.devin.ai/sessions/59777c74e423429aa9889a61d08cd1e3

@BerriAI BerriAI deleted a comment from devin-ai-integration Bot Jul 24, 2026
@tin-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 42aba4f. Configure here.

@tin-berri
tin-berri merged commit 2798f39 into litellm_internal_staging Jul 24, 2026
79 checks passed
@tin-berri
tin-berri deleted the litellm_lit4748_autorouter_logs branch July 24, 2026 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants