feat(cli): add lite autoroute to QA complexity-based auto-routing against a real proxy

litellm/proxy/client/cli/README.md

@@ -489,6 +489,65 @@ This is a one-time file patch and restore, not a live traffic interceptor. A Cla
 
 Cursor is not supported: it has no equivalent file-based config to hot-patch this way, since its model routing lives in its own app storage and is configured through its GUI.
 
+### QA Complexity-Based Auto-Routing Against Your Real Proxy
+
+`lite autoroute` lets you try LiteLLM's complexity-based auto-routing -- picking a cheaper or more expensive model depending on how complex a prompt looks -- against models your key already has access to on your real, running proxy, without editing that proxy's `config.yaml` and without any real request ever bypassing it. It builds a second, throwaway proxy locally that forwards every request back to your real proxy, and points Claude Code at that local proxy for the duration of the session.
+
+#### List Your Accessible Model Groups
+
+```bash
+lite model-groups list [--format table|json]
+```
+
+Lists the model groups your key can reach on the proxy, via `/model_group/info`, along with each group's mode (`chat`, `embedding`, etc.) and per-token pricing. This is also what `lite autoroute configure` uses internally to discover what it can offer you.
+
+#### Configure the Auto-Router
+
+```bash
+lite autoroute configure
+```
+
+An interactive wizard. It runs the same model-group discovery as above, splits the results into chat-capable and embedding-capable pools, and asks you to assign one or more models from the chat pool to each of the four complexity tiers -- SIMPLE, MEDIUM, COMPLEX, REASONING. Each tier's picker is a type-to-filter fuzzy search (fzf-style) rather than a scrollable numbered list, so it stays usable even with hundreds of model groups: type a substring to narrow the list, tab to toggle a model into the selection, enter to confirm (assigning more than one model to a tier is exactly when this matters -- complexity_router picks randomly among a tier's pool per request, and adaptive mode specifically depends on having more than one candidate to choose from). From there it optionally offers: classifying prompt complexity with an LLM (again picked from your discovered pool) instead of the free built-in heuristic scorer, semantic keyword matching for tier assignment (needs an embedding model from the pool), and adaptive (bandit-based) selection layered on top of tiering.
+
+The wizard writes the result to `~/.litellm/autorouter/config.yaml` with `0600` permissions, since the file embeds your real proxy API key. Every model referenced anywhere in that config -- tier targets, the classifier model, the embedding model -- becomes its own `litellm_proxy/<model-name>` deployment whose `api_base` and `api_key` point back at your real proxy. That is the trick that keeps your real proxy's config untouched: every actual network call this generates, whether it is the routed completion, an LLM-classifier call, or an embedding call, forwards transparently through your real, already-running proxy with your real key.
+
+You do not need to tell Claude Code to request `autorouter` by name yourself: `lite autoroute up` also sets `ANTHROPIC_DEFAULT_SONNET_MODEL`, `ANTHROPIC_DEFAULT_HAIKU_MODEL`, and `ANTHROPIC_DEFAULT_OPUS_MODEL` to `autorouter` in `~/.claude/settings.json`, so every one of Claude Code's own model tiers requests it directly regardless of `/model` or whatever it defaults to otherwise. (A bare `model_name: "*"` deployment looks like the obvious way to catch any request instead, but litellm's Router looks up auto-router deployments by the literal requested model string with no wildcard resolution, so a `"*"` entry would never actually match real traffic -- these env var overrides are what makes it work.)
+
+You must run `configure` at least once before `up`; running `up` first fails with a clear error telling you to configure first.
+
+#### Launch the Ephemeral Auto-Router Proxy
+
+```bash
+lite autoroute up
+```
+
+Starts a local, throwaway litellm proxy on a random free port, running the config `configure` generated, with a freshly-minted random API key baked in for this session only (your real proxy key never leaves the generated config -- it only appears there, forwarding to your real proxy). It waits for the ephemeral proxy to report healthy, then patches `~/.claude/settings.json` the same way `lite up` does, except with a static `ANTHROPIC_AUTH_TOKEN` env var instead of an `apiKeyHelper`, since this key is short-lived and self-issued rather than something needing SSO refresh. Any `claude` session started afterward, from any terminal, routes through the ephemeral proxy.
+
+`lite autoroute up` runs in the foreground and streams the ephemeral proxy's own log file into your terminal, so you can watch its routing decisions -- which tier and model got picked for each request -- as you use Claude Code normally. Press Ctrl-C (or send SIGTERM) to stop it; this kills the child proxy process and restores your original Claude Code settings, in that order.
+
+#### Recover From an Unclean Shutdown
+
+```bash
+lite autoroute down
+```
+
+If the `lite autoroute up` process dies uncleanly -- `kill -9`, a crash -- rather than being stopped with Ctrl-C, `down` is the manual recovery path: it kills any leftover ephemeral proxy process found via a recorded pid file and restores Claude Code's settings from whatever backup is on disk.
+
+#### Example
+
+```bash
+lite autoroute configure
+lite autoroute up
+# use Claude Code as normal in another terminal; routing decisions stream live
+lite autoroute down   # only needed if `up` was killed uncleanly instead of Ctrl-C'd
+```
+
+#### Caveats
+
+Adaptive mode's learned state does not persist across `lite autoroute up` sessions -- there is no local database, so every session starts adaptive selection cold. A Claude Code session already running before `up` started, or still running when it stops, keeps whatever settings it loaded at its own startup; like `lite up`, this is a one-time file patch and restore, not a live traffic interceptor. Only Claude Code is supported, for the same reason as `lite up`: no other supported agent (for example Cursor) has an equivalent hot-patchable config file.
+
+A session that outlives `up` (or is still running the moment you stop it) keeps sending requests, master key included, to that now-freed loopback port until you restart it. Once the ephemeral proxy process exits, nothing stops another local account on the same machine from binding that same port and receiving those requests instead -- unlike `lite up`'s `apiKeyHelper`, which is re-resolved per request, `autoroute`'s master key is a static value, so whoever receives them gets a live-looking token along with the prompt content. Restart any Claude Code session before you consider the machine clean, run `lite autoroute down` promptly rather than leaving a stopped session's settings patched, and do not run `lite autoroute up` on a shared or multi-tenant host.
+
 ## Environment Variables
 
 The CLI respects the following environment variables:

litellm/proxy/client/cli/commands/autoroute/commands.py

@@ -0,0 +1,167 @@
+import atexit
+import json
+import secrets
+import signal
+import threading
+from types import FrameType
+
+import click
+import yaml
+from pydantic import JsonValue, TypeAdapter
+
+from ..up import CLAUDE_SETTINGS_PATH, load_json_or_empty, restore_claude_settings, write_backup
+from ..up import BackupRecord as ClaudeBackupRecord
+from .process import (
+    AUTOROUTE_DIR,
+    CONFIG_PATH,
+    LOG_PATH,
+    PidRecord,
+    ProcessLaunchError,
+    allocate_free_port,
+    clear_pid_record,
+    is_running,
+    launch_proxy,
+    poll_liveliness,
+    read_pid_record,
+    secure_create,
+    stream_log,
+    terminate,
+    write_pid_record,
+)
+from .settings import merge_claude_settings_static_token
+from .wizard import run_configure_wizard
+
+AUTOROUTE_BACKUP_PATH = AUTOROUTE_DIR / "claude_settings_backup.json"
+
+_GENERATED_CONFIG_ADAPTER = TypeAdapter(dict[str, JsonValue])
+
+
+def _mint_and_embed_master_key() -> str:
+    """Generate a fresh key for this session and write it into the generated config.yaml.
+
+    Must go under general_settings, not litellm_settings -- the proxy server only ever
+    reads general_settings.master_key (proxy_server.py:4530) to authenticate requests. A
+    key placed under litellm_settings is silently ignored, leaving the ephemeral proxy with
+    no real auth: any request reaches it regardless of the token Claude Code sends.
+    """
+    master_key = secrets.token_urlsafe(32)
+    with open(CONFIG_PATH, "r") as f:
+        generated = _GENERATED_CONFIG_ADAPTER.validate_python(yaml.safe_load(f))
+    general_settings = generated.get("general_settings")
+    updated_settings: dict[str, JsonValue] = {
+        **(general_settings if isinstance(general_settings, dict) else {}),
+        "master_key": master_key,
+    }
+    updated: dict[str, JsonValue] = {**generated, "general_settings": updated_settings}
+    with secure_create(CONFIG_PATH) as f:
+        yaml.safe_dump(updated, f, sort_keys=False)
+    return master_key
+
+
+@click.group(name="autoroute")
+def autoroute_group() -> None:
+    """QA complexity-based auto-routing against models your key can already use"""
+
+
+@autoroute_group.command("configure")
+@click.pass_context
+def configure(ctx: click.Context) -> None:
+    """Discover accessible models and generate an ephemeral auto-router config"""
+    run_configure_wizard(ctx)
+
+
+@autoroute_group.command("up")
+def up() -> None:
+    """Launch the ephemeral auto-router proxy and route Claude Code through it"""
+    if not CONFIG_PATH.exists():
+        raise click.ClickException("No config found. Run `lite autoroute configure` first.")
+
+    existing_pid = read_pid_record()
+    if existing_pid is not None and is_running(existing_pid.pid):
+        raise click.ClickException(
+            "An ephemeral proxy is already running (lite autoroute up looks already active). "
+            "Run `lite autoroute down` first."
+        )
+
+    if AUTOROUTE_BACKUP_PATH.exists():
+        raise click.ClickException(
+            f"{AUTOROUTE_BACKUP_PATH} already exists -- `lite autoroute up` looks like it's already "
+            "running (or crashed without cleanup). Run `lite autoroute down` first."
+        )
+
+    master_key = _mint_and_embed_master_key()
+    port = allocate_free_port()
+    base_url = f"http://127.0.0.1:{port}"
+    process = launch_proxy(CONFIG_PATH, port, LOG_PATH)
+    write_pid_record(PidRecord(pid=process.pid, port=port, config_path=str(CONFIG_PATH), log_path=str(LOG_PATH)))
+
+    try:
+        poll_liveliness(base_url, LOG_PATH, process)
+    except ProcessLaunchError as e:
+        terminate(process.pid)
+        clear_pid_record()
+        raise click.ClickException(str(e))
+
+    original_existed = CLAUDE_SETTINGS_PATH.exists()
+    original_settings = load_json_or_empty(CLAUDE_SETTINGS_PATH)
+    write_backup(
+        ClaudeBackupRecord(existed=original_existed, content=original_settings if original_existed else None),
+        AUTOROUTE_BACKUP_PATH,
+    )
+    merged = merge_claude_settings_static_token(original_settings, base_url, master_key)
+    CLAUDE_SETTINGS_PATH.parent.mkdir(parents=True, exist_ok=True)
+    with secure_create(CLAUDE_SETTINGS_PATH) as f:
+        json.dump(merged, f, indent=2)
+
+    click.echo(f"litellm: ephemeral auto-router proxy up at {base_url} (pid {process.pid})")
+    click.echo("Claude Code sessions started now will route through it. Press Ctrl-C to stop and restore.")
+
+    stop_event = threading.Event()
+    restored = threading.Lock()
+
+    def _teardown() -> None:
+        if not restored.acquire(blocking=False):
+            return
+        terminate(process.pid)
+        clear_pid_record()
+        restore_claude_settings(CLAUDE_SETTINGS_PATH, AUTOROUTE_BACKUP_PATH)
+        click.echo("\nStopped ephemeral proxy and restored Claude Code settings.")
+        click.echo(
+            f"Restart any Claude Code session still open from this session, or another local account could "
+            f"bind the now-free port {port} and receive its requests. Do not use `lite autoroute up` on a "
+            f"shared or multi-tenant host."
+        )
+
+    def _handle_signal(_signum: int, _frame: FrameType | None) -> None:
+        stop_event.set()
+
+    signal.signal(signal.SIGINT, _handle_signal)
+    signal.signal(signal.SIGTERM, _handle_signal)
+    atexit.register(_teardown)
+
+    log_thread = threading.Thread(target=stream_log, args=(LOG_PATH, stop_event), daemon=True)
+    log_thread.start()
+
+    stop_event.wait()
+    _teardown()
+
+
+@autoroute_group.command("down")
+def down() -> None:
+    """Restore Claude Code settings and stop a leftover ephemeral proxy, if any"""
+    record: PidRecord | None = read_pid_record()
+    if record is not None and is_running(record.pid):
+        terminate(record.pid)
+        click.echo(f"Stopped leftover ephemeral proxy (pid {record.pid}).")
+    clear_pid_record()
+
+    restored = restore_claude_settings(CLAUDE_SETTINGS_PATH, AUTOROUTE_BACKUP_PATH)
+    if restored is None:
+        click.echo("Nothing to restore.")
+    elif restored.existed:
+        click.echo(f"Restored {CLAUDE_SETTINGS_PATH} to its original contents.")
+    else:
+        click.echo(f"Removed {CLAUDE_SETTINGS_PATH} (it did not exist before `lite autoroute up`).")
+
+
+__all__ = ["autoroute_group"]