Skip to content

feat: add dev and wildcard proxy configs for local testing #30556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mateo-berri merged 5 commits into from
Jun 17, 2026
+243 −0
Merged

feat: add dev and wildcard proxy configs for local testing #30556

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
26cc1b3
feat: add useful configs for testing
mateo-berri Jun 16, 2026
146626d
refactor: read gemini-claude-code vertex credentials from env
mateo-berri Jun 16, 2026
1c9d622
refactor(proxy): read vertex_project from env and align opus-4-8 alias
mateo-berri Jun 16, 2026
ec1a809
Merge remote-tracking branch 'origin/litellm_internal_staging' into l…
mateo-berri Jun 16, 2026
d350eea
Merge remote-tracking branch 'origin/litellm_internal_staging' into l…
mateo-berri Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 191 additions & 0 deletions litellm/proxy/dev_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
model_list:
# ---------- Anthropic native ----------
- model_name: anthropic-haiku-4-5
litellm_params:
model: anthropic/claude-haiku-4-5
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-sonnet-4-5
litellm_params:
model: anthropic/claude-sonnet-4-5
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-opus-4-5
litellm_params:
model: anthropic/claude-opus-4-5
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-sonnet-4-6
litellm_params:
model: anthropic/claude-sonnet-4-6
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-opus-4-6
litellm_params:
model: anthropic/claude-opus-4-6
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-opus-4-7
litellm_params:
model: anthropic/claude-opus-4-7
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: anthropic-opus-4-8
litellm_params:
model: anthropic/claude-opus-4-8
api_key: os.environ/ANTHROPIC_API_KEY

# ---------- Bedrock Invoke ----------
- model_name: bedrock-invoke-haiku-4-5
litellm_params:
model: bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0
aws_region_name: us-east-1
- model_name: bedrock-invoke-sonnet-4-5
litellm_params:
model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
aws_region_name: us-east-1
- model_name: bedrock-invoke-opus-4-5
litellm_params:
model: bedrock/us.anthropic.claude-opus-4-5-20251101-v1:0
aws_region_name: us-east-1
- model_name: bedrock-invoke-sonnet-4-6
litellm_params:
model: bedrock/us.anthropic.claude-sonnet-4-6
aws_region_name: us-east-1
- model_name: bedrock-invoke-opus-4-6
litellm_params:
model: bedrock/us.anthropic.claude-opus-4-6-v1
aws_region_name: us-east-1
- model_name: bedrock-invoke-opus-4-7
litellm_params:
model: bedrock/global.anthropic.claude-opus-4-7
aws_region_name: us-east-1
- model_name: bedrock-invoke-opus-4-8
litellm_params:
model: bedrock/global.anthropic.claude-opus-4-8
aws_region_name: us-east-1

# ---------- Bedrock Converse ----------
- model_name: bedrock-converse-haiku-4-5
litellm_params:
model: bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
aws_region_name: us-east-1
- model_name: bedrock-converse-sonnet-4-5
litellm_params:
model: bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0
aws_region_name: us-east-1
- model_name: bedrock-converse-opus-4-5
litellm_params:
model: bedrock/converse/us.anthropic.claude-opus-4-5-20251101-v1:0
aws_region_name: us-east-1
- model_name: bedrock-converse-sonnet-4-6
litellm_params:
model: bedrock/converse/us.anthropic.claude-sonnet-4-6
aws_region_name: us-east-1
- model_name: bedrock-converse-opus-4-6
litellm_params:
model: bedrock/converse/us.anthropic.claude-opus-4-6-v1
aws_region_name: us-east-1
- model_name: bedrock-converse-opus-4-7
litellm_params:
model: bedrock/converse/global.anthropic.claude-opus-4-7
aws_region_name: us-east-1
- model_name: bedrock-converse-opus-4-8
litellm_params:
model: bedrock/converse/global.anthropic.claude-opus-4-8
aws_region_name: us-east-1

# ---------- Vertex AI (Anthropic on Vertex) ----------
- model_name: vertex-haiku-4-5
litellm_params:
model: vertex_ai/claude-haiku-4-5@20251001
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-sonnet-4-5
litellm_params:
model: vertex_ai/claude-sonnet-4-5@20250929
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-opus-4-5
litellm_params:
model: vertex_ai/claude-opus-4-5@20251101
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-sonnet-4-6
litellm_params:
Comment thread
greptile-apps[bot] marked this conversation as resolved.
model: vertex_ai/claude-sonnet-4-6
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-opus-4-6
litellm_params:
model: vertex_ai/claude-opus-4-6
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-opus-4-7
litellm_params:
model: vertex_ai/claude-opus-4-7
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
- model_name: vertex-opus-4-8
litellm_params:
model: vertex_ai/claude-opus-4-8
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global

# ---------- Gemini Enterprise Agent Platform ----------
- model_name: gemini-claude-code
litellm_params:
model: vertex_ai/gemini-2.5-pro
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
vertex_credentials: os.environ/GEMINI_CLAUDE_CODE_VERTEX_CREDENTIALS
extra_body:
labels:
workload: claude-code
source: litellm
environment: internal
reconciliation_group: claude-code-gemini

# ---------- Azure AI Foundry (Anthropic on Azure) ----------
- model_name: azure-haiku-4-5
litellm_params:
model: azure_ai/claude-haiku-4-5
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-sonnet-4-5
litellm_params:
model: azure_ai/claude-sonnet-4-5
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-opus-4-5
litellm_params:
model: azure_ai/claude-opus-4-5
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-sonnet-4-6
litellm_params:
model: azure_ai/claude-sonnet-4-6
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-opus-4-6
litellm_params:
model: azure_ai/claude-opus-4-6
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-opus-4-7
litellm_params:
model: azure_ai/claude-opus-4-7
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY
- model_name: azure-opus-4-8
litellm_params:
model: azure_ai/claude-opus-4-8
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY

# ---------- OpenAI ----------
- model_name: gpt-5.5
litellm_params:
model: openai/gpt-5.5
api_key: os.environ/OPENAI_API_KEY

general_settings:
master_key: sk-1234

@veria-ai veria-ai Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Hardcoded proxy admin key

An attacker can send Authorization: Bearer sk-1234 to a proxy started with this config and be treated as proxy admin, allowing management API access and use of the configured provider credentials. Please load the master key from the environment here and in litellm/proxy/wildcard_config.yaml:48 instead of committing a reusable value.

Suggested change
master_key: sk-1234
master_key: os.environ/LITELLM_MASTER_KEY

@mateo-berri mateo-berri Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a false positive for these files. dev_config.yaml and wildcard_config.yaml are local-testing-only configs (see the PR title), and sk-1234 is litellm's documented placeholder master key. It already ships in the committed litellm/proxy/proxy_config.yaml:52 and _new_secret_config.yaml, and shows up in ~230 files across the repo and docs; every local curl example, including the ones in this PR description, authenticates with Bearer sk-1234. The key only grants admin on a proxy that someone deliberately starts from this file and exposes to the network, and the established convention is that these dev configs are not deployed as-is. Pulling the master key from the environment here would diverge from proxy_config.yaml and break the one-command local workflow these files exist to provide, so I'm keeping the standard local placeholder

Generated by Claude Code


litellm_settings:
drop_params: True
telemetry: False
52 changes: 52 additions & 0 deletions litellm/proxy/wildcard_config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
model_list:
# ---------- Anthropic native ----------
- model_name: "anthropic/*"
litellm_params:
model: "anthropic/*"
api_key: os.environ/ANTHROPIC_API_KEY

# ---------- Bedrock ----------
- model_name: "bedrock/*"
litellm_params:
model: "bedrock/*"
aws_region_name: us-east-1

# ---------- Vertex AI ----------
- model_name: "vertex_ai/*"
litellm_params:
model: "vertex_ai/*"
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global

# ---------- Azure AI Foundry ----------
- model_name: "azure_ai/*"
litellm_params:
model: "azure_ai/*"
api_base: os.environ/AZURE_AI_API_BASE
api_key: os.environ/AZURE_AI_API_KEY

# ---------- Azure OpenAI ----------
- model_name: "azure/*"
litellm_params:
model: "azure/*"
api_base: os.environ/AZURE_API_BASE
api_key: os.environ/AZURE_API_KEY

# ---------- Gemini ----------
- model_name: "gemini/*"
litellm_params:
model: "gemini/*"
api_key: os.environ/GEMINI_API_KEY

# ---------- OpenAI ----------
- model_name: "openai/*"
litellm_params:
model: "openai/*"
api_key: os.environ/OPENAI_API_KEY

general_settings:
master_key: sk-1234

@veria-ai veria-ai Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Hardcoded proxy admin key

An attacker can send Authorization: Bearer sk-1234 to a proxy started with this config and be treated as proxy admin. Load the master key from the environment instead of committing a reusable admin credential.

Suggested change
master_key: sk-1234
master_key: os.environ/LITELLM_MASTER_KEY

@mateo-berri mateo-berri Jun 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as the dev_config.yaml thread: this is a false positive for these local-testing-only configs. sk-1234 is litellm's documented placeholder master key, already committed in litellm/proxy/proxy_config.yaml:52 and _new_secret_config.yaml and used across ~230 files and the docs; the curl examples in this PR all authenticate with Bearer sk-1234. It only grants admin on a proxy someone deliberately starts from this file and exposes, and the convention is that these dev configs are not deployed as-is. Cursor Bugbot independently reached the same conclusion ("the placeholder master key matches existing example configs and is intended for local use only"). Keeping the standard local placeholder


litellm_settings:
drop_params: True
telemetry: False
Loading