Skip to content

fix(ci): let the install smoke test boot its key-less proxy config - #42296

Merged
mateo-berri merged 1 commit into
mainfrom
litellm_ci_smoke_test_master_key
Sep 21, 2026
Merged

mateo-berri merged 1 commit into
mainfrom
litellm_ci_smoke_test_master_key

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • the smoke test passes LITELLM_DANGEROUSLY_PERMIT_WEAK_OR_UNSET_MASTER_KEY=true to the proxy it starts
  • the yaml and the boot check stay as they are
  • one helper change covers all three jobs and local runs

User Flow

Before: a contributor pushes any commit to main or a run-ci PR and CircleCI reports the three installing_litellm_on_python jobs red, although their change touched nothing near the proxy's install

  1. They push a commit and open the workflow at https://app.circleci.com/pipelines/github/BerriAI/litellm and see installing_litellm_on_python, installing_litellm_on_python_3_13, and installing_litellm_on_python_v2_migration_resolver red
  2. They open the red job's "Run tests" step (job 2195208, for example) and see FAILED tests/local_testing/test_basic_python_version.py::test_litellm_proxy_server_config_no_general_settings - Failed: Failed to connect to the server
  3. Above it, the proxy the test started printed LiteLLM proxy refused to start: no master key is set, so every request would be accepted without authentication and exited, so the test's GET http://localhost:4000/health/liveliness got connection refused
  4. They rerun the job and get the same result, since every run boots the same key-less test config

After: the same push turns the three jobs green

  1. They push a commit and open the workflow at https://app.circleci.com/pipelines/github/BerriAI/litellm and see the three installing jobs green
  2. They open the job's "Run tests" step and see test_litellm_proxy_server_config_no_general_settings PASSED and 7 passed, 1 deselected
  3. In the same step the proxy the test started logs its startup, the test's GET http://localhost:4000/health/liveliness returns 200 "I'm alive!", and its POST http://localhost:4000/chat/completions with Authorization: Bearer 1234567890 returns 200
  4. A proxy started outside this test with no master key still refuses to boot with the same message, so nothing about the check itself changed

Relevant issues

Follow-up to #42019 (the boot check). #42120 fixed the same class for the Google proxy fixture by giving its config a real key; this test cannot take that shape, since it asserts a made-up bearer token is accepted by the key-less proxy and ten other tests share its yaml, so it uses the documented local-dev override in the child's env instead

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The smoke test hardcodes http://localhost:4000 and the proxy it starts reads the repo's .env, so both runs happened in a throwaway Linux container that replays the CircleCI job as is: python 3.12, uv 0.10.9 and rust 1.98.0 installed the way .circleci/config.yml does (sha256-checked), the repo copied in without .env, .git, or .venv, and only the env names the CircleCI project also defines (OPENAI_API_KEY, AZURE_AI_API_KEY, AZURE_AI_API_BASE). No LITELLM_* variable is set anywhere in the container; the run prints that before pytest starts. Install step, identical on both sides:

$ uv sync --frozen --all-groups --all-extras --python 3.12
...
 + zstandard==0.25.0

The three CircleCI jobs on this PR's tip are the x86_64 run of the same thing; their links are in the After section.

Before (9a90ada)

  1. env | grep -E "^LITELLM_" | cut -d= -f1 || echo "(none)" printed (none), and ss -ltnp | grep ":4000 " printed (none)

  2. uv run --no-sync python -m pytest -vv tests/local_testing/test_basic_python_version.py -k "not legacy_resolver"

  3. Observed: the proxy the test started exited before the health check, with this in its stderr

    litellm.proxy.auth.master_key_boot_check.UnsafeMasterKeyError: LiteLLM proxy refused to start: no master key is set, so every request would be accepted without authentication. The fix is printed once the server exits.
    LiteLLM proxy refused to start: no master key is set, so every request would be accepted without authentication.
    Neither general_settings.master_key nor the LITELLM_MASTER_KEY environment variable is set.
    
  4. Observed: the test's GET http://localhost:4000/health/liveliness got connection refused

    FAILED tests/local_testing/test_basic_python_version.py::test_litellm_proxy_server_config_no_general_settings - Failed: Failed to connect to the server
    ============ 1 failed, 6 passed, 1 deselected in 164.17s (0:02:44) =============
    

After (a8745f2)

  1. env | grep -E "^LITELLM_" | cut -d= -f1 || echo "(none)" printed (none), and ss -ltnp | grep ":4000 " printed (none)

  2. uv run --no-sync python -m pytest -vv tests/local_testing/test_basic_python_version.py -k "not legacy_resolver"

  3. Observed: the proxy stayed up, GET http://localhost:4000/health/liveliness returned 200 "I'm alive!", and POST http://localhost:4000/chat/completions with Authorization: Bearer 1234567890 returned 200 (a real gpt-3.5-turbo call through the test's test_openai_models deployment)

    tests/local_testing/test_basic_python_version.py::test_litellm_proxy_server PASSED
    tests/local_testing/test_basic_python_version.py::test_litellm_proxy_server_config_no_general_settings PASSED
    ================= 7 passed, 1 deselected in 1092.68s (0:18:12) =================
    
  4. CircleCI on this tip, x86_64 cimg/python: installing_litellm_on_python, installing_litellm_on_python_3_13, and installing_litellm_on_python_v2_migration_resolver are all green, where the same three jobs were red on every main pipeline since 2026-09-20

Type

🐛 Bug Fix
✅ Test

Caveats (if any)

Low

  • no new test file: the smoke test itself is the regression check, and it runs on every run-ci build
  • the env var name is a literal, like the other harnesses that set it, since this file must not import litellm at collection time
  • the yaml stays key-less on purpose: ten other tests share it and each sets the override in its own process
  • the local proof ran on an arm64 image instead of CI's x86_64 cimg/python:3.12; the CI jobs on this PR's tip are the x86_64 run
  • local_testing_part1 is red on this tip on test_get_model_info_bedrock_cross_region_capability_parity and test_get_model_info_bedrock_models (supports_audio_input missing from two Bedrock cost map rows). Main's own pipeline 89979 (job 2196579) fails the same two tests, so it is fleet-wide and not from this PR, which touches no cost map row
  • llm_translation_testing is red on this tip on ten tests/llm_translation/test_fireworks_ai_translation.py tests; main's pipeline 89979 (job 2196570) fails the same ten, so it is fleet-wide and not from this PR
  • test_bad_database_url is red on this tip with Expected error not found. Test failed. after the proxy printed Prisma's P1001: Can't reach database server; main's pipeline 89979 (job 2196551) fails the same way with the same output, so it is fleet-wide and not from this PR, which touches no proxy code
  • unit is red on this tip on twelve tests/unit/router_strategy/complexity_router/test_jev_classifier.py tests (RuntimeError: <asyncio.locks.Event> is bound to a different event loop from GLOBAL_LOGGING_WORKER.flush()). Those tests landed on main in feat(auto-router): add JEV classifier alongside LLM classifier #41886 (a83773c, six minutes before this branch's merge base 9a90ada), no main pipeline has run a tip containing it yet (the latest, 89979, is at cc1a315), and the three other branches whose merge base contains it (pipelines 895046cf, 62b51c91, 16e36045) fail the same twelve while every branch based before it fails none. This PR touches no unit test and nothing the unit job runs

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

  • a8745f2 passes /live-pr-risk

The install smoke test starts the proxy on test_config_no_auth.yaml, which has no master key on purpose, and #42019's boot check now refuses that, so the three installing_litellm_on_python jobs have been red on main since 2026-09-20. Pass the documented local-dev override to the proxy child so the test keeps its no-auth config and the boot check stays as it is
@devin-ai-integration

devin-ai-integration Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge because the override is confined to the intended key-less test subprocess and matches the proxy startup guard

Summary

This PR lets the installation smoke test start its intentionally key-less proxy configuration after the new master-key boot check

  • Adds the narrowly scoped startup override only to the proxy subprocess environment
  • Preserves the existing liveness and chat-completion smoke assertions
  • Does not change production proxy behavior

Reviews (1) · Last reviewed commit: "fix(ci): let the install smoke test boot..."

@codecov

codecov Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit a8745f2. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit 13c604c into main Sep 21, 2026
138 of 146 checks passed
@mateo-berri
mateo-berri deleted the litellm_ci_smoke_test_master_key branch September 21, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant