feat: litellm config yaml - #369
Conversation
|
Warning Review limit reached
Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughLiteLLM now uses a repository-relative configuration mount and a new configuration file defining runtime settings, provider model mappings, smart routing, environment-backed credentials, and an optional guardrail example. ChangesLiteLLM configuration
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅MegaLinter analysis: Success
Notices📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining See detailed reports in MegaLinter artifacts Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@agent/compose.yaml`:
- Line 116: Update the bind-mount entry in the Compose service to use the
absolute host source path /apps/litellm/config.yaml while retaining
/app/config.yaml:ro as the container destination and read-only mode.
In `@agent/litellm/config.yaml`:
- Around line 46-57: Declare NOUS_API_KEY= in agent/.env.example, then update
the LiteLLM service environment configuration to require both NOUS_API_KEY and
OPENCODE_ZEN_API_KEY using the ${VAR:?} fail-fast syntax. Ensure the
smart-router credentials resolve through the existing litellm credential
configuration without changing routing behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a6f80e5f-4cdc-4d59-abe7-77d8c61faf00
📒 Files selected for processing (2)
agent/compose.yamlagent/litellm/config.yaml
| read_only: true | ||
| volumes: | ||
| - /apps/litellm/config.yaml:/app/config.yaml:ro | ||
| - ./litellm/config.yaml:/app/config.yaml:ro |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Keep the bind-mount source under /apps/litellm/.
./litellm/config.yaml violates the required host bind-mount layout. Provision the repository config to /apps/litellm/config.yaml and retain that absolute source path.
Proposed fix
- - ./litellm/config.yaml:/app/config.yaml:ro
+ - /apps/litellm/config.yaml:/app/config.yaml:roAs per coding guidelines, Docker Compose files must “Use host bind mounts under /apps/<service-name>/ instead of named volumes.”
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - ./litellm/config.yaml:/app/config.yaml:ro | |
| - /apps/litellm/config.yaml:/app/config.yaml:ro |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@agent/compose.yaml` at line 116, Update the bind-mount entry in the Compose
service to use the absolute host source path /apps/litellm/config.yaml while
retaining /app/config.yaml:ro as the container destination and read-only mode.
Source: Coding guidelines
| - model_name: smart-router | ||
| litellm_params: | ||
| model: auto_router/complexity_router | ||
| complexity_router_config: | ||
| tiers: | ||
| SIMPLE: opencode-zen/deepseek-v4-flash-free | ||
| MEDIUM: opencode-zen/deepseek-v4-flash-free | ||
| COMPLEX: nous/hy3:free | ||
| REASONING: nous/hy3:free | ||
| adaptive: true | ||
| session_affinity: true | ||
| complexity_router_default_model: nous/hy3:free |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C2 'NOUS_API_KEY|OPENCODE_ZEN_API_KEY' \
agent/.env.example agent/compose.yaml agent/litellm/config.yamlRepository: yxtay/docker-stacks
Length of output: 993
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files agent | sed -n '1,200p'
echo "== relevant env/compose/config excerpts =="
for f in agent/.env.example agent/compose.yaml agent/litellm/config.yaml agent/Dockerfile* Dockerfile* compose.yaml; do
if [ -f "$f" ]; then
echo "--- $f"
wc -l "$f"
sed -n '1,140p' "$f"
fi
doneRepository: yxtay/docker-stacks
Length of output: 6835
🌐 Web query:
LiteLLM config.yaml model custom_base_url os.environ API_KEY credential_values api_key os.environ/NOUS_API_KEY
💡 Result:
In LiteLLM, you can use the os.environ/ prefix within your config.yaml to dynamically load values from environment variables [1][2][3]. This syntax is supported for various configuration fields, ensuring sensitive credentials like API keys are not hardcoded in your configuration files [4][5]. Key mechanisms for using environment variables and custom configurations include: 1. Dynamic Loading with os.environ/ You can reference any environment variable by using the prefix os.environ/ followed by the variable name [3]. When LiteLLM parses the config.yaml, it will call os.getenv() to retrieve the value [2]. Example usage in model_list: model_list: - model_name: my-model litellm_params: model: openai/gpt-4 api_key: os.environ/NOUS_API_KEY # Loads from environment variable NOUS_API_KEY api_base: os.environ/CUSTOM_API_BASE_URL # Example of using it for custom base URL 2. Centralized Credential Management To avoid duplication, you can define shared credentials using the credential_list section and reference them in your model_list using litellm_credential_name [6][7]. Example: model_list: - model_name: my-azure-model litellm_params: model: azure/gpt-4 litellm_credential_name: my_shared_creds credential_list: - credential_name: my_shared_creds credential_values: api_key: os.environ/AZURE_API_KEY api_base: os.environ/AZURE_API_BASE credential_info: description: "Shared credentials" 3. Custom API Base URL The api_base parameter within litellm_params allows you to point to non-default endpoints, such as local gateways (e.g., vLLM) or specific provider regions [8][2]. This field also supports the os.environ/ syntax, enabling flexible configuration across different environments [5]. 4. Important Note on Server Root Path If your goal is to change the base URL path of the LiteLLM Proxy service itself (e.g., serving at /api/v1 instead of the root), you should set the SERVER_ROOT_PATH environment variable in your environment, rather than modifying config.yaml [9]. Never commit your config.yaml to version control if it contains plain-text secrets [4][5]. Always prefer the os.environ/ method to keep credentials secure [4].
Citations:
- 1: https://docs.litellm.ai/docs/proxy/configs
- 2: https://github.com/BerriAI/litellm/blob/main/docs/my-website/docs/proxy/configs.md
- 3: (feat) - allow using os.environ/ vars for any value on config.yaml BerriAI/litellm#6276
- 4: https://berriai-litellm.mintlify.app/proxy/configs
- 5: https://berriai-litellm.mintlify.app/proxy/quick-start
- 6: https://github.com/BerriAI/litellm/blob/v1.82.6.dev2/docs/my-website/docs/proxy/configs.md
- 7: https://github.com/BerriAI/litellm/blob/4dc9726d/docs/my-website/docs/proxy/configs.md
- 8: https://gist.github.com/yigitkonur/4053c1e31f9351da1d91a0ca410442fd
- 9: https://docs.litellm.ai/docs/proxy/custom_root_ui
Declare and fail fast on the smart-router credentials.
smart-router routes to nous/hy3:free for COMPLEX, REASONING, and fallback requests, and litellm_credential_name: nous resolves its key from NOUS_API_KEY. Add NOUS_API_KEY= to agent/.env.example and require both NOUS_API_KEY and OPENCODE_ZEN_API_KEY in the LiteLLM litellm service with ${VAR:?} so an unset credential fails startup rather than causing request failure.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@agent/litellm/config.yaml` around lines 46 - 57, Declare NOUS_API_KEY= in
agent/.env.example, then update the LiteLLM service environment configuration to
require both NOUS_API_KEY and OPENCODE_ZEN_API_KEY using the ${VAR:?} fail-fast
syntax. Ensure the smart-router credentials resolve through the existing litellm
credential configuration without changing routing behavior.
Source: Coding guidelines
1 participant
Summary by CodeRabbit
New Features
Bug Fixes