Skip to content

test: stop the zai tests from leaking env and litellm globals into the session - #37834

Merged
yuneng-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_zai_test_env_isolation
Aug 22, 2026
Merged

test: stop the zai tests from leaking env and litellm globals into the session#37834
yuneng-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_zai_test_env_isolation

Conversation

@yuneng-berri

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • zai tests leave LITELLM_LOCAL_MODEL_COST_MAP set for the whole session
  • they also leave litellm.model_cost swapped to the local map
  • disable_aiohttp_transport is set with no restore at all

How it solves it:

  • all three go through monkeypatch, which restores on teardown
  • cost-map setup becomes a fixture the five tests that need it request

User Flow

Before: a proxy admin debugging a cost discrepancy asks a maintainer to confirm a price, and the maintainer's local run reports a stale one

  1. The maintainer runs the unit tier in one process, zai included
  2. A later test in that same process reads litellm.model_cost and gets the checked-in local snapshot, not the live map
  3. Any price the maintainer quotes from that run reflects the snapshot, so the answer they send back can disagree with what the proxy actually charged

After: the same run reads the real map, so the answer matches the proxy

  1. The maintainer runs the same unit tier in one process, zai included
  2. The zai tests put LITELLM_LOCAL_MODEL_COST_MAP and litellm.model_cost back when they finish
  3. A later test reads the live map, so a quoted price matches what the proxy charged

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

The leak is only observable from outside the pytest process, so the probe runs the file in-process and reads the environment back afterwards. LITELLM_LOCAL_MODEL_COST_MAP is not part of the conftest snapshot, so nothing else restores it.

# leakprobe.py
import os, litellm, pytest
os.environ.pop("LITELLM_LOCAL_MODEL_COST_MAP", None)
pytest.main(["tests/test_litellm/llms/zai/test_zai_provider.py", "-q", "-p", "no:randomly"])
print("LITELLM_LOCAL_MODEL_COST_MAP after:", repr(os.environ.get("LITELLM_LOCAL_MODEL_COST_MAP")))

Before (ff02d5c)

  1. python leakprobe.py
  2. LITELLM_LOCAL_MODEL_COST_MAP after: 'True'

After (b87f1bb)

  1. python leakprobe.py
  2. LITELLM_LOCAL_MODEL_COST_MAP after: None

Type

🧹 Refactoring
✅ Test

Caveats (if any)

  • litellm.model_cost restores via the conftest snapshot either way

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…e session

test_zai_provider.py set LITELLM_LOCAL_MODEL_COST_MAP and litellm.model_cost
directly and never put them back, so every test that ran after it in the same
process saw a local cost map instead of the real one. The two respx tests did
the same to litellm.disable_aiohttp_transport with no restore at all.

Both now go through monkeypatch, which restores on teardown including when the
test fails. The cost-map setup moves into a fixture requested by exactly the
five tests that read the cost map.
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR prevents Z.AI tests from leaking selected environment and LiteLLM global state by moving mutations into pytest monkeypatch-managed fixtures.

  • Adds a local cost-map fixture and applies it to the five pricing and capability tests that require the checked-in map.
  • Makes async-transport overrides teardown-safe in the completion tests.
  • Ratchets the test-quality budgets by the exact number of eliminated direct mutations.

Confidence Score: 5/5

The PR appears safe to merge, with the changed test globals restored at fixture teardown and no actionable regression identified.

The five cost-map tests retain their original setup and assertions while pytest now restores the environment and model-cost global, and both transport overrides are similarly restored; the quality-budget changes match the eliminated violations.

Important Files Changed

Filename Overview
tests/test_litellm/llms/zai/test_zai_provider.py Replaces persistent environment and module-global assignments with function-scoped monkeypatch restoration without weakening the existing assertions.
test-quality-budget.json Lowers TQ004 by five and TQ005 by seven, exactly matching the direct mutations removed from the Z.AI tests.

Reviews (1): Last reviewed commit: "test: stop the zai tests from leaking en..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…itellm_zai_test_env_isolation

# Conflicts:
#	test-quality-budget.json
#	tests/test_litellm/llms/zai/test_zai_provider.py
…itellm_zai_test_env_isolation

# Conflicts:
#	test-quality-budget.json
@yuneng-berri
yuneng-berri merged commit 3ac339c into litellm_internal_staging Aug 22, 2026
66 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_zai_test_env_isolation branch August 22, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants