feat: litellm-master-key - #378
Conversation
|
Warning Review limit reached
Next review available in: 50 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughLiteLLM configuration examples now include sample key values, and router fallback mappings support ordered lists of fallback candidates across reasoning tiers. ChangesLiteLLM Configuration
Estimated code review effort: 1 (Trivial) | ~5 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅MegaLinter analysis: Success
Notices📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining See detailed reports in MegaLinter artifacts Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@agent/.env.example`:
- Around line 19-21: Update the LITELLM_MASTER_KEY example in the environment
template to be empty or a clearly non-credential placeholder, and document that
each deployment must provide its own generated value. Keep the existing
LITELLM_SALT_KEY guidance unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 02f03cf4-f74e-4e34-a07a-b4dea4e6da24
📒 Files selected for processing (2)
agent/.env.exampleagent/litellm/config.yaml
| LITELLM_MASTER_KEY=sk-litellm-master-key | ||
| # generate using `openssl rand -hex 32` | ||
| LITELLM_MASTER_KEY= | ||
| LITELLM_SALT_KEY= |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== candidate files =="
git ls-files | rg '(^|/)(\.env\.example|compose\.ya?ml)$' || true
echo
echo "== agent/.env.example =="
if [ -f agent/.env.example ]; then
nl -ba agent/.env.example | sed -n '1,80p'
fi
echo
echo "== agent compose references =="
if [ -f agent/compose.yaml ]; then
nl -ba agent/compose.yaml | sed -n '1,220p'
elif [ -f agent/docker-compose.yml ]; then
nl -ba agent/docker-compose.yml | sed -n '1,220p'
elif [ -f agent/docker-compose.yaml ]; then
nl -ba agent/docker-compose.yaml | sed -n '1,220p'
else
fd -i 'compose.*ya?ml' agent -x sh -c 'echo "--- $1"; nl -ba "$1" | sed -n "1,220p"' sh {}
fi
echo
echo "== LITELLM_MASTER_KEY / LITELLM_SALT_KEY usages =="
rg -n -H 'LITELLM_(MASTER_|SALT_)KEY|master_key|salt' . --glob '!node_modules' --glob '!dist' --glob '!build' | sed -n '1,200p'Repository: yxtay/docker-stacks
Length of output: 847
🌐 Web query:
LiteLLM master_key setting documentation
💡 Result:
The master_key in LiteLLM is the proxy's administrative credential [1][2][3]. It is used to authenticate admin API calls, serve as the password for the Admin UI, and manage virtual keys [4][2][5][3]. Configuration You can set the master key using either an environment variable or the configuration file [4][2]: 1. Environment Variable: Set the LITELLM_MASTER_KEY variable (must start with sk-) [4][5][3]. 2. Config File: Specify it under general_settings: master_key in your config.yaml [4][6]. If both are set, the value in the config file takes precedence [2]. Important Considerations - Security: The master key bypasses standard model-level checks and gives access to every model on the proxy [1]. It should be treated as a highly sensitive operator credential [1]. - Encryption: In some deployments, if a separate LITELLM_SALT_KEY is not configured, the master key is also used to encrypt sensitive data (such as API keys and credentials) at rest in the database [2]. If the master key is used for this purpose, rotating it requires a specific re-encryption process to prevent losing access to stored credentials [2]. - Best Practices: It is recommended to keep the key in a secure secret manager and follow the official master key rotation documentation when updating it [2][3].
Citations:
- 1: https://docs.litellm.ai/docs/proxy/key_auth_arch
- 2: https://docs.litellm.ai/docs/proxy/master_key_rotations
- 3: https://docs.litellm.ai/docs/proxy/prod
- 4: https://docs.litellm.ai/docs/proxy/virtual_keys
- 5: https://docs.litellm.ai/docs/proxy/docker_quick_start
- 6: https://docs.litellm.ai/docs/proxy/configs
🏁 Script executed:
#!/bin/bash
set -euo pipefail
sed -n '1,120p' agent/.env.example
echo
sed -n '1,220p' agent/compose.yaml
echo
printf '\nLITELLM usages in agent:\n'
rg -n -H 'LITELLM_(MASTER_|SALT_)KEY|master_key|salt' agent --glob '!node_modules' --glob '!dist' --glob '!build' || trueRepository: yxtay/docker-stacks
Length of output: 6002
🌐 Web query:
LiteLLM v1.94.0 environment variables LITELLM_MASTER_KEY configured
💡 Result:
In LiteLLM, the LITELLM_MASTER_KEY environment variable serves as the primary administrative key for the LiteLLM Proxy [1][2][3]. It grants full admin access, allowing you to perform operations such as creating, blocking, or unblocking virtual keys [1][3][4]. Key details regarding the LITELLM_MASTER_KEY: Configuration and Precedence - Environment Variable: You can set the master key directly via the environment variable LITELLM_MASTER_KEY (e.g., export LITELLM_MASTER_KEY="sk-...") [2][3]. - Configuration File: You can also specify the master key in your config.yaml file under the general_settings section using master_key: <key> or master_key: os.environ/LITELLM_MASTER_KEY [1][3]. - Precedence: If both are configured, the value defined in general_settings.master_key in the config.yaml file generally takes precedence over the environment variable [5]. Operational Usage - Authentication: The master key is used to authenticate requests to administrative endpoints, such as the key management APIs (/key/generate, /key/block, /key/unblock) [1][4]. It is passed in the Authorization header as a Bearer token: Authorization: Bearer LITELLM_MASTER_KEY [1]. - Admin UI: The master key acts as the password for the Admin UI when accessing the proxy [6]. - Security Best Practices: It is highly recommended to use a strong, unique key starting with sk- [1][3]. Always avoid hardcoding the key in configuration files; use environment variables or secret management systems (e.g., AWS Secret Manager) instead [3][7]. - Versioning Note: LiteLLM v1.94.0 includes updates to proxy behavior, including fixes related to master key routing and startup initialization, ensuring that the master key configuration is preserved correctly during the proxy's lifecycle [8][9].
Citations:
- 1: https://docs.litellm.ai/docs/proxy/virtual_keys
- 2: https://mintlify.wiki/BerriAI/litellm/api/environment-variables
- 3: https://mintlify.wiki/BerriAI/litellm/deployment/security
- 4: https://github.com/BerriAI/litellm/blob/main/docs/my-website/docs/proxy/virtual_keys.md
- 5: https://docs.litellm.ai/docs/proxy/master_key_rotations
- 6: https://docs.litellm.ai/docs/proxy/docker_quick_start
- 7: https://docs.litellm.ai/docs/secret_managers/aws_secret_manager
- 8: https://github.com/BerriAI/litellm/releases/tag/v1.94.0
- 9: fix(proxy): preserve master_key set by initialize() during lifespan startup BerriAI/litellm#22483
Do not commit the default LiteLLM master key.
agent/compose.yaml loads .env into the LiteLLM service, and LITELLM_MASTER_KEY is the admin/proxy credential. Copying this example unchanged can deploy the shared placeholder sk-litellm-master-key; leave it empty or a non-credential placeholder and require generated per-deployment values.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@agent/.env.example` around lines 19 - 21, Update the LITELLM_MASTER_KEY
example in the environment template to be empty or a clearly non-credential
placeholder, and document that each deployment must provide its own generated
value. Keep the existing LITELLM_SALT_KEY guidance unchanged.
✅MegaLinter analysis: Success
Notices📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining See detailed reports in MegaLinter artifacts Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining
|
✅
|
| Descriptor | Linter | Files | Fixed | Errors | Warnings | Elapsed time |
|---|---|---|---|---|---|---|
| ✅ ACTION | actionlint | 4 | 0 | 0 | 0.23s | |
| ✅ ACTION | zizmor | 4 | 0 | 0 | 0 | 0.29s |
| ✅ BASH | bash-exec | 11 | 0 | 0 | 0.03s | |
| ✅ BASH | shellcheck | 11 | 0 | 0 | 0.16s | |
| ✅ BASH | shfmt | 11 | 0 | 0 | 0 | 0.02s |
| jscpd | yes | 1 | no | 0.1s | ||
| ✅ EDITORCONFIG | editorconfig-checker | 107 | 0 | 0 | 0.17s | |
| ✅ JSON | prettier | 1 | 0 | 0 | 0 | 0.21s |
| ✅ JSON | v8r | 1 | 0 | 0 | 1.72s | |
| ✅ MARKDOWN | markdownlint | 5 | 0 | 0 | 0 | 0.81s |
| ✅ MARKDOWN | markdown-table-formatter | 5 | 0 | 0 | 0 | 0.16s |
| ✅ REPOSITORY | betterleaks | yes | no | no | 1.5s | |
| ✅ REPOSITORY | checkov | yes | no | no | 27.08s | |
| ✅ REPOSITORY | gitleaks | yes | no | no | 0.62s | |
| ✅ REPOSITORY | git_diff | yes | no | no | 0.01s | |
| ✅ REPOSITORY | grype | yes | no | no | 65.05s | |
| ✅ REPOSITORY | osv-scanner | yes | no | no | 0.23s | |
| ✅ REPOSITORY | secretlint | yes | no | no | 1.26s | |
| ✅ REPOSITORY | syft | yes | no | no | 2.03s | |
| ✅ REPOSITORY | trivy | yes | no | no | 11.5s | |
| ✅ REPOSITORY | trivy-sbom | yes | no | no | 0.28s | |
| ✅ REPOSITORY | trufflehog | yes | no | no | 6.59s | |
| lychee | 56 | 41 | 0 | 1.72s | ||
| ✅ YAML | prettier | 49 | 0 | 0 | 0 | 1.09s |
| ✅ YAML | v8r | 49 | 0 | 0 | 17.73s | |
| ✅ YAML | yamllint | 49 | 0 | 0 | 1.18s |
Detailed Issues
⚠️ COPYPASTE / jscpd - 1 error
Using config from /action/lib/.automation/.jscpd.json
Clone found (bash)
- bin/oci-rm-stack-create.sh [32:1 - 40:5] (9 lines, 51 tokens)
bin/oci-rm-stack-update.sh [12:1 - 20:5]
┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ bash │ 7 │ 214 │ 1120 │ 1 │ 8 (3.74%) │ 51 (4.55%) │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ hcl │ 4 │ 382 │ 1407 │ 0 │ 0 (0.00%) │ 0 (0.00%) │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ txt │ 1 │ 29 │ 379 │ 0 │ 0 (0.00%) │ 0 (0.00%) │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total: │ 12 │ 625 │ 2906 │ 1 │ 8 (1.28%) │ 51 (1.75%) │
└────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 1 clones.
HTML report saved to megalinter-reports/copy-paste/jscpd-report.html
ERROR: jscpd found too many duplicates (1.3%) over threshold (0.0%)
time: 28.097ms
⚠️ SPELL / lychee - 41 errors
📝 Summary
---------------------
🔍 Total...........68
🔗 Unique..........58
✅ Successful......20
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........0
❓ Unknown..........0
🚫 Errors..........41
⛔ Unsupported.....41
Errors in agent/compose.yaml
[ERROR] http://localhost:2375/_ping (at 89:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:4000/health/liveliness (at 129:70) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9119/api/status (at 41:33) | Connection refused - server may be down or port blocked
[ERROR] https://hermes/ (at 12:67) | Connection failed. Check network connectivity and firewall settings
Errors in agent/litellm/config.yaml
[ERROR] http://headroom:8787/ (at 119:19) | Connection failed. Check network connectivity and firewall settings
[404] https://inference-api.nousresearch.com/v1 (at 99:17) | Rejected status code: 404 Not Found
[404] https://opencode.ai/zen/go/v1 (at 111:17) | Rejected status code: 404 Not Found
[404] https://opencode.ai/zen/v1 (at 105:17) | Rejected status code: 404 Not Found
Errors in arcane/compose.yaml
[ERROR] http://localhost:3552/ (at 6:27) | Connection refused - server may be down or port blocked
Errors in homeassistant/compose.yaml
[ERROR] http://localhost:10000/health (at 146:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:6052/version (at 105:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8095/ (at 66:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8123/ (at 27:33) | Connection refused - server may be down or port blocked
Errors in immich/compose.yaml
[ERROR] http://localhost:8080/ (at 160:32) | Connection refused - server may be down or port blocked
Errors in infra/compose.yaml
[ERROR] http://localhost:2375/_ping (at 60:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9002/healthz (at 95:36) | Connection refused - server may be down or port blocked
Errors in monitoring/compose.yaml
[ERROR] http://localhost:8090/ (at 73:44) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8090/ (at 94:27) | Connection refused - server may be down or port blocked
Errors in pangolin/compose.yaml
[ERROR] http://gerbil:3004/ (at 63:23) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://localhost/ping (at 121:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3001/api/v1/ (at 51:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3004/healthz (at 87:32) | Connection refused - server may be down or port blocked
[ERROR] http://pangolin:3001/api/v1/ (at 65:24) | Connection failed. Check network connectivity and firewall settings
Errors in pangolin/traefik/dynamic/config.yml
[ERROR] http://pangolin:3000/ (at 80:18) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://pangolin:3002/ (at 85:18) | Connection failed. Check network connectivity and firewall settings
Errors in pangolin/traefik/traefik.template.yml
[ERROR] http://pangolin:3001/api/v1/traefik-config (at 7:15) | Connection failed. Check network connectivity and firewall settings
Errors in proxy/compose.yaml
[ERROR] http://localhost/healthz (at 36:32) | Connection refused - server may be down or port blocked
[ERROR] https://tinyauth/ (at 50:24) | Connection failed. Check network connectivity and firewall settings
Errors in security/compose.yaml
[ERROR] http://localhost:8080/ (at 52:16) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8080/health (at 34:32) | Connection refused - server may be down or port blocked
Errors in torrent/compose.yaml
[ERROR] http://localhost:6868/ (at 220:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:7878/ping (at 140:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8191/health (at 16:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8989/ping (at 182:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9696/ping (at 45:33) | Connection refused - server may be down or port blocked
[ERROR] https://profilarr/ (at 205:15) | Connection failed. Check network connectivity and firewall settings
Errors in usenet/compose.yaml
[ERROR] http://127.0.0.1:7000/ (at 83:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:1337/ (at 288:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3000/health (at 26:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:7000/health (at 134:32) | Connection refused - server may be down or port blocked
[ERROR] https://aiostreams/ (at 232:18) | Connection failed. Check network connectivity and firewall settings
Hint: You can configure accepted/rejected response codes with `-a` or `--accept`
Notices
📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)
See detailed reports in MegaLinter artifacts
Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)
- Documentation: Custom Flavors
- Command:
npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters ACTION_ACTIONLINT,ACTION_ZIZMOR,BASH_EXEC,BASH_SHELLCHECK,BASH_SHFMT,COPYPASTE_JSCPD,EDITORCONFIG_EDITORCONFIG_CHECKER,JSON_V8R,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_GITLEAKS,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,SPELL_LYCHEE,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R
Show us your support by starring ⭐ the repository
1 participant
Summary by CodeRabbit