Skip to content

fix(ui): keep untouched stored auto-router booleans and reminder marker casing on save - #42703

Merged
yuneng-berri merged 1 commit into
mainfrom
litellm_autorouter_save_drops_managed_booleans
Sep 23, 2026
Merged

yuneng-berri merged 1 commit into
mainfrom
litellm_autorouter_save_drops_managed_booleans

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Saving an untouched auto-router drops route_housekeeping_to_cheapest_tier: true
  • The same save drops max_tokens_from_tier_model: true
  • reminder_markers come back lowercased without the user touching them

How it solves it:

  • Save keeps the stored value for those keys when the form still equals what was hydrated from it
  • An actual edit to any of them still wins and goes through the builder as before
  • Regression test asserts the whole rebuilt config equals the stored one

User Flow

Before: an admin opens a router created before #42293, saves without changing anything, and two settings silently disappear

  1. They open http://localhost:3000/models-and-endpoints/, click Auto-Routers, then click legacy-auto-router
  2. On the Raw JSON tab they see "route_housekeeping_to_cheapest_tier": true, "max_tokens_from_tier_model": true and "open": "<System-Reminder>"
  3. They click Overview, then Edit Auto Router, scroll to the bottom and click Save Changes without editing any field
  4. The modal closes with a success toast
  5. They open the Raw JSON tab again: both boolean keys are gone and the marker now reads "open": "<system-reminder>"
  6. GET http://localhost:4000/model/info confirms the same: both keys are null and the markers are lowercased

After: the same untouched save leaves the stored settings exactly as they were

  1. They open http://localhost:3000/models-and-endpoints/, click Auto-Routers, then click legacy-auto-router
  2. On the Raw JSON tab they see "route_housekeeping_to_cheapest_tier": true, "max_tokens_from_tier_model": true and "open": "<System-Reminder>"
  3. They click Overview, then Edit Auto Router, scroll to the bottom and click Save Changes without editing any field
  4. The modal closes with a success toast
  5. They open the Raw JSON tab again: both keys are still true and the marker still reads "open": "<System-Reminder>"
  6. GET http://localhost:4000/model/info returns the same three values unchanged

Relevant issues

Follow-up to #42293, which added these three keys to the managed set

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally: npx vitest run src/components/edit_auto_router src/components/add_model from ui/litellm-dashboard (44 files, 1159 tests)
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Shared setup: proxy on http://localhost:4000 with STORE_MODEL_IN_DB=True and Postgres, dashboard via npm run dev in ui/litellm-dashboard on http://localhost:3000, logged in as admin with the master key. The router was seeded through the API the way a pre-#42293 config would sit in the DB, then reseeded to the identical state before the After leg:

curl -s -X POST http://localhost:4000/model/new -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d '{
  "model_name": "legacy-auto-router",
  "litellm_params": {
    "model": "auto_router/complexity_router",
    "complexity_router_default_model": "gpt-4o-mini",
    "complexity_router_config": {
      "tiers": {"SIMPLE": ["gpt-4o-mini"], "MEDIUM": ["gpt-4o-mini"], "COMPLEX": ["gpt-4o-mini"], "REASONING": ["gpt-4o-mini"]},
      "default_model": "gpt-4o-mini",
      "classifier_type": "heuristic_first",
      "heuristic_first_max_tier": "SIMPLE",
      "classifier_llm_config": {"model": "gpt-4o-mini", "timeout_ms": 3000},
      "route_housekeeping_to_cheapest_tier": true,
      "housekeeping_patterns": ["conversation title"],
      "reminder_markers": [{"open": "<System-Reminder>", "close": "</System-Reminder>"}],
      "max_tokens_from_tier_model": true
    }
  }
}'

Read-back used after each Save:

curl -s -H "Authorization: Bearer $LITELLM_MASTER_KEY" http://localhost:4000/model/info |
  jq '.data[] | select(.model_name=="legacy-auto-router") | .litellm_params.complexity_router_config
      | {route_housekeeping_to_cheapest_tier, max_tokens_from_tier_model, reminder_markers}'

Reviewer steps, identical on both legs: open http://localhost:3000/models-and-endpoints/, click Auto-Routers, click legacy-auto-router, click Edit Auto Router, scroll to the bottom of the modal, click Save Changes without touching any field, then click the Raw JSON tab and run the read-back curl

Before (40ec84c)

Untouched Save on a pre-#42293 router

  1. Edit Auto Router modal, nothing edited, about to click Save Changes

before modal

  1. Raw JSON after Save: route_housekeeping_to_cheapest_tier and max_tokens_from_tier_model are gone, marker is <system-reminder>

before raw json

  1. Read-back curl output
{
  "route_housekeeping_to_cheapest_tier": null,
  "max_tokens_from_tier_model": null,
  "reminder_markers": [
    {
      "open": "<system-reminder>",
      "close": "</system-reminder>"
    }
  ]
}

Regression tests

  1. npx vitest run src/components/edit_auto_router/build_updated_complexity_router_config.test.ts with the new tests on unfixed code
 × returns the stored config unchanged when nothing was edited
 × keeps both housekeeping and max-token booleans stored as true
 × keeps the stored reminder marker casing
AssertionError: expected { tiers: { …(4) }, …(39) } to deeply equal { tiers: { …(4) }, …(41) }
-   "max_tokens_from_tier_model": true,
-       "close": "</System-Reminder>",
-       "open": "<System-Reminder>",
+       "close": "</system-reminder>",
+       "open": "<system-reminder>",
-   "route_housekeeping_to_cheapest_tier": true,
AssertionError: expected undefined to be true // Object.is equality
AssertionError: expected [ { open: '<system-reminder>', …(1) } ] to deeply equal [ { open: '<System-Reminder>', …(1) } ]
 Test Files  1 failed (1)
      Tests  3 failed | 96 passed (99)

After (3fbbf57)

Untouched Save on a pre-#42293 router

  1. Edit Auto Router modal, nothing edited, about to click Save Changes

after modal

  1. Raw JSON after Save: both keys still true, marker still <System-Reminder>

after raw json

  1. Read-back curl output
{
  "route_housekeeping_to_cheapest_tier": true,
  "max_tokens_from_tier_model": true,
  "reminder_markers": [
    {
      "open": "<System-Reminder>",
      "close": "</System-Reminder>"
    }
  ]
}

Regression tests

  1. Same vitest command on the fixed code
 Test Files  1 passed (1)
      Tests  99 passed (99)
  1. Mutation proof: reverting edit_auto_router_modal.tsx to the merge-base version while keeping the new tests
 Test Files  1 failed (1)
      Tests  3 failed | 96 passed (99)
  1. Mutation proof: deleting only the if (unedited.has(key)) return false; guard
 Test Files  1 failed (1)
      Tests  4 failed | 95 passed (99)
  1. Existing suites unedited: build_complexity_router_config.test.ts, hydrate_complexity_router_config.test.ts, edit_auto_router_modal.integration.test.tsx and the rest of build_updated_complexity_router_config.test.ts
 Test Files  4 passed (4)
      Tests  455 passed (455)

Pre-push gates: the base-vs-head A/B above is the live-pr-risk check for this change (the only dependent of buildUpdatedComplexityRouterConfig is the modal's Save and availability check, both exercised in the run), and an adversarial five-lens self-review of the diff ran inline with no finding at or above the blocker threshold

Type

🐛 Bug Fix

Caveats (if any)

Medium

  • A stored value hydration rejects (say a string where a boolean belongs) is now preserved on untouched saves instead of being dropped
    • Before this change the managed set always rebuilt those keys, so invalid stored values were silently cleaned up on Save
    • Editing the field still replaces it

Low

  • UNEDITED_STORED_VALUE_KEYS lists the three keys the builder emits only on false; a fourth such key would need adding here too
  • Equality is by JSON.stringify, so a form value that differs only in object key order counts as edited and falls back to the old builder path
  • npx tsc --noEmit in the dashboard reports the same pre-existing errors at the merge base and at this tip, none in the changed files

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/c9c9660a39f94254a359d117f6644a08
Open in Devin Desktop: https://app.devin.ai/desktop/session/c9c9660a39f94254a359d117f6644a08?variant=devin

…er casing on save

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 23, 2026 07:38
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge because untouched stored values are preserved without changing the established behavior for edited values

Summary

This PR preserves three stored auto-router settings when the corresponding hydrated form values remain untouched

  • Keeps explicit stored boolean values from disappearing during an unchanged save
  • Preserves the original casing of untouched reminder markers
  • Continues routing edited values through the existing configuration builder
  • Adds regression coverage for untouched round trips and explicit edits

Reviews (1) · Last reviewed commit: "fix(ui): keep untouched stored auto-rout..."

@yuneng-berri
yuneng-berri merged commit 04b1b07 into main Sep 23, 2026
88 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_autorouter_save_drops_managed_booleans branch September 23, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant