Skip to content

feat(cli): fixed sk-1234 master key and debug flags for lite autoroute up - #33651

Closed
devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_autoroute_up_debuggability
Closed

devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_autoroute_up_debuggability

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Captured live at commit eb9154d545e9a7f2d46ade32594685058c02ba8c, hitting the real Anthropic API

uv run --no-sync lite autoroute up --detailed-debug
litellm: ephemeral auto-router proxy up at http://127.0.0.1:54197 (pid 11080)
Config: /home/ubuntu/.litellm/autorouter/config.yaml
Log: /home/ubuntu/.litellm/autorouter/proxy.log
Port: 54197
Master key: sk-1234
Claude Code sessions started now will route through it. Press Ctrl-C to stop and restore.

Real completion against the ephemeral proxy with the fixed key

curl -sS -w '\nHTTP_STATUS:%{http_code}\n' 'http://127.0.0.1:54197/v1/chat/completions' \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"model":"claude","messages":[{"role":"user","content":"hi"}]}'
{"id":"chatcmpl-a21d6132-fa32-45fc-8e20-83d0d124ab12","model":"claude","choices":[{"finish_reason":"stop","message":{"content":"# Hello! How can I help you today?","role":"assistant"}}],"usage":{"completion_tokens":17,"prompt_tokens":8,"total_tokens":25}}
HTTP_STATUS:200

A wrong key is still rejected

curl -sS -w '\nHTTP_STATUS:%{http_code}\n' -X POST 'http://127.0.0.1:54197/global/spend/reset' \
  -H 'Authorization: Bearer wrong-key' -H 'Content-Type: application/json' -d '{}'
{"error":{"message":"Authentication Error, Tried to access route=/global/spend/reset, which is only for MASTER KEY","type":"auth_error","code":"401"}}
HTTP_STATUS:401

~/.litellm/autorouter/proxy.log confirms the flag reached the proxy subprocess

proxy_server.py:901 - worker_config: ... "debug": false, "detailed_debug": true, ... "config": "/home/ubuntu/.litellm/autorouter/config.yaml"

A second lite autoroute up after a clean SIGTERM teardown printed Master key: sk-1234 again, so the key no longer changes between runs

Type

🆕 New Feature

Changes

lite autoroute up used to mint a random secrets.token_urlsafe(32) master key on every run, which made it painful to curl the local proxy while debugging. Since this is a local dev CLI, the key is now always the fixed sk-1234; _mint_and_embed_master_key becomes _embed_master_key and still writes it under general_settings.master_key in the generated config

To make the command easier to debug, up now prints the generated config path, log file path, port, and master key at startup, and grows --debug / --detailed-debug flags that are forwarded to the litellm.proxy.proxy_cli subprocess as --debug / --detailed_debug via new keyword-only params on launch_proxy

Tests pin the fixed key end to end (generated config, ANTHROPIC_AUTH_TOKEN in Claude settings, startup output) and assert the debug flags propagate to the subprocess argv. The CLI README section for lite autoroute up is updated to match

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/531f86a2375b45f6985488c598f1f06d
Requested by: @krrish-berri-2

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@krrish-berri-2 krrish-berri-2 self-assigned this Jul 17, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes two changes to the lite autoroute up CLI command: it replaces the per-session random master key with the fixed development key sk-1234, and it adds --debug / --detailed-debug flags that are forwarded to the proxy subprocess. Startup output is also extended to print the config path, log path, port, and key.

  • The fixed sk-1234 key eliminates the need to look up the generated key before curling the local proxy, but it also means any local process that knows this convention can authenticate to the ephemeral proxy without reading the 0o600-protected settings file. The proxy still binds to 127.0.0.1 only, so exposure is loopback-scoped.
  • test_embeds_master_key_under_general_settings in test_config.py now uses \"sk-1234\" as both the function argument and the expected value, reducing its ability to catch a future regression where the function ignores its master_key parameter.

Confidence Score: 4/5

Safe to merge for a local dev CLI; the fixed key is a deliberate convenience trade-off and the proxy is loopback-only.

The core logic — embedding the key in the generated config, forwarding debug flags to the subprocess, printing startup info — is straightforward and well-tested. The main open question is whether sk-1234 is an acceptable credential for a tool that forwards to real API endpoints; the loopback binding mitigates network exposure but removes the file-permission barrier that previously protected the credential. The test weakening in test_config.py is minor but worth fixing before the next change touches that function.

test_config.py (test sentinel value) and commands.py (fixed key decision) deserve a second look.

Security Review

  • Universally guessable proxy credential (commands.py:37): The ephemeral proxy's master key is changed from a cryptographically random per-session token to the fixed string sk-1234. Any local process that knows this convention can now authenticate to the ephemeral proxy (which forwards requests to real Anthropic API endpoints) without needing to read the ~/.claude/settings.json file. The proxy binds to 127.0.0.1 only, limiting exposure to the local machine. The previous design required an attacker to read the 0o600-protected settings file to discover the key.

Important Files Changed

Filename Overview
litellm/proxy/client/cli/commands/autoroute/commands.py Replaces random per-session master key with hardcoded sk-1234, adds --debug/--detailed-debug flags forwarded to the subprocess, and prints config/log/port/key at startup; key change trades local security isolation for dev convenience.
litellm/proxy/client/cli/commands/autoroute/process.py Adds debug and detailed_debug keyword-only parameters to launch_proxy, appending --debug/--detailed_debug to the subprocess command tuple; refactor is clean and the file-handle scope is unchanged.
tests/test_litellm/proxy/client/cli/autoroute/test_commands.py Removes secrets.token_urlsafe monkeypatches, pins assertions to sk-1234, adds test_passes_debug_flags_to_proxy verifying launch_proxy receives debug/detailed_debug kwargs; coverage is good.
tests/test_litellm/proxy/client/cli/autoroute/test_config.py Changes test_embeds_master_key_under_general_settings to pass "sk-1234" and assert "sk-1234" back — using the same literal for input and expected output weakens the test's ability to detect a regression where the function ignores its parameter.
tests/test_litellm/proxy/client/cli/autoroute/test_process.py Adds test_forwards_debug_flags that patches subprocess.Popen and verifies --debug and --detailed_debug appear at the end of the command args; clear and correct.
litellm/proxy/client/cli/README.md Updates the lite autoroute up description to reflect the fixed sk-1234 key and new --debug/--detailed-debug flags; documentation is accurate and matches the code changes.

Reviews (1): Last reviewed commit: "feat(cli): improve autoroute up debuggab..." | Re-trigger Greptile

AUTOROUTE_BACKUP_PATH = AUTOROUTE_DIR / "claude_settings_backup.json"

_GENERATED_CONFIG_ADAPTER = TypeAdapter(dict[str, JsonValue])
MASTER_KEY = "sk-1234"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 security Well-known key lowers bar for lateral local API access

Replacing the per-session random key with the fixed sk-1234 means any process running on the same machine — including scripts, CI runners, or other user-space tools — that is aware of this convention can authenticate to the ephemeral proxy and make real Anthropic API calls on the developer's behalf without reading ~/.claude/settings.json. The previous random key required reading that file (permissions 0o600) to learn the credential; sk-1234 requires no file access at all. The proxy does bind only to 127.0.0.1, and the teardown message already cautions against multi-tenant hosts, so this is a deliberate trade-off for dev convenience. However, it's worth considering a compromise — e.g. a short fixed prefix like sk-local- followed with a few random bytes — that retains discoverability while avoiding a universally guessable credential.

Comment on lines 172 to +175
def test_embeds_master_key_under_general_settings(self):
config = _base_config()
proxy_config = build_generated_proxy_config(config, "sk-master-123")
assert proxy_config["general_settings"] == {"master_key": "sk-master-123"}
proxy_config = build_generated_proxy_config(config, "sk-1234")
assert proxy_config["general_settings"] == {"master_key": "sk-1234"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Test no longer catches key-parameter regression

build_generated_proxy_config still accepts a master_key parameter, but the test now passes "sk-1234" and asserts "sk-1234" back — the same literal for both input and expected output. If a future change hardcodes MASTER_KEY inside build_generated_proxy_config instead of using the argument, this test would still pass and the regression would go undetected. Using a distinct value (e.g. "sk-sentinel-test-key") for the test argument would ensure the function's parameter is actually plumbed through.

Rule Used: What: Flag any modifications to existing tests and... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_autoroute_up_debuggability (eb9154d) with litellm_internal_staging (9cae6fa)

Open in CodSpeed

@yuneng-berri
yuneng-berri deleted the branch litellm_internal_staging September 13, 2026 04:31
@yuneng-berri yuneng-berri reopened this Sep 13, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor

Closing this since #42019 now refuses to boot on sk-1234 and #42011 scrubbed it from shipped configs, so the hardcoded key cannot land

@mateo-berri
mateo-berri deleted the litellm_autoroute_up_debuggability branch September 20, 2026 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants