Repository navigation
test(unit): run tests/unit with only an allowlisted host environment - #43299
yuneng-berri wants to merge 1 commit into
Conversation
CircleCI's unit job inherits every project env var, so real provider keys, REDIS_HOST, DATABASE_URL and AWS or Azure credentials reached tests that assume none are set. Locally, litellm's import-time load_dotenv did the same from any .env up the tree. The unit conftest now drops every variable outside a small allowlist and disables dotenv before litellm is imported.
|
Folded into #43294, which now carries every stale-test and CI-hygiene fix in one place, so closing this duplicate |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
| for _name in tuple(os.environ): | ||
| if _name not in HOST_ENVIRONMENT_ALLOWLIST and not _name.startswith(HOST_ENVIRONMENT_ALLOWED_PREFIXES): | ||
| del os.environ[_name] |
There was a problem hiding this comment.
Legacy MCP peer setting removed
The mcp-integration CI shard supplies MCP_TEST_PEER_PYTHON to select its separately installed MCP 1.28.1 interpreter. This loop deletes that setting, so the MCP e2e test falls back to sys.executable and starts the peer with the suite’s MCP 2.2.0 environment. The shard no longer tests the intended legacy-peer compatibility and may fail against the wrong SDK.
| for _name in tuple(os.environ): | ||
| if _name not in HOST_ENVIRONMENT_ALLOWLIST and not _name.startswith(HOST_ENVIRONMENT_ALLOWED_PREFIXES): | ||
| del os.environ[_name] |
There was a problem hiding this comment.
TLDR
Problem this solves:
unitfails on main from CircleCI's own project env vars.envanywhere up the tree leaks in the same wayHow it solves it:
tests/unit/conftest.pydrops every env var outside a small allowlistPYTHON_DOTENV_DISABLEDbefore litellm is importedPre-Submission checklist
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Shared setup: every one of the 95 CircleCI project env var names set to a fake value (
REDIS_PORTnumeric,DATABASE_URLa URL), plus a.envat the repo root withXAI_API_KEY,REDIS_HOSTandREDIS_PORT, then the modules that fail on main run the way theunitjob runs themBefore (1e6c983)
14 failed, 1162 passed: the xAI OAuth tests pick up the real key, the Redis sentinel tests pick upREDIS_HOST, the S3 and Azure token tests pick up cloud credentialsunitjob fails the same tests (job 2218776 and the 06:09 UTC run at 99655b6)After (1b3fab9)
1175 passed, 0 failedunder the same env and.envunitjob on this PR is the full-suite checkType
Test
Infrastructure
Caveats (if any)
Medium
unitred is env leakagetest_proxy_serverauth, uvicorn access log and video logging tests pass alone even with the CI env, so they look like cross-module state leaks under xdist; this PR's CI run shows what remainsLow
monkeypatchinside the test insteadFinal Attestation