test: enforce F811 so a duplicate definition cannot silently replace the first - #37878
Merged
ryan-crabbe-berri merged 1 commit intoAug 21, 2026
Conversation
…the first A name bound twice keeps only the second binding. In `tests/` that is nearly always a repeated import, harmless but misleading, and the same rule is what catches the cases that are not harmless: a local that shadows an import the module still calls, and a second `def test_x` that quietly replaces the first. 311 of the 344 sites were repeated imports and came out with ruff's own fix. The remaining 33 needed a decision. Four modules imported a name they never used because a local definition below already shadowed it. Two comprehensions bound `call` over `unittest.mock.call`, which those modules import and use. One test rebound the two module handles its nested reload closure had captured. One class attribute shadowed an unused `status` import. The load-test fixtures move to a conftest, which is how pytest is meant to share them, so the test module no longer imports three fixture names it never calls. The nine `prisma_client` parameters keep a narrow `noqa`: pytest resolves that fixture by name before the body runs, so the parameter never shadows anything.
Contributor
|
Too many files changed for review (172 files, 100 file limit). Bypass the limit by tagging |
yuneng-berri
approved these changes
Aug 21, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
6 tasks
yuneng-berri
added a commit
that referenced
this pull request
Aug 22, 2026
* test: unshadow the module handles the F811 sweep left behind, and pin the two live tests that went red with it The F811 sweep in #37878 removed the fixture-local `import litellm` from four conftests, but the bare `import litellm.proxy.proxy_server` a few lines below still binds `litellm` as a function local, so `importlib.reload(litellm)` runs before the name is assigned and every test in those directories errors at setup. The `hasattr` guard on the line above already proves the module is loaded, so the import only ever bound the name. Drop it, and enable F823 in ruff-tests.toml, which flags all four sites at the failing line and would have blocked the sweep The same sweep renamed the `check_non_streaming_response` parameter but left one read of `completion`, which now resolves to `litellm.completion`, and removed an import whose side effect was the only thing making `litellm.proxy.proxy_server` reachable in the moderation hook test. That test already takes `monkeypatch`, so patch the router through it and stop leaking the router into later tests `test_content_policy_exception_openai` passed vacuously until #37887 turned it into a real `pytest.raises`, and OpenAI no longer rejects a lyrics prompt with a content policy error. Inject an AsyncOpenAI client whose transport answers with OpenAI's own `content_policy_violation` rejection so the mapping to ContentPolicyViolationError is exercised every run `test_async_create_batch` hit a 409 cancelling a batch OpenAI had already marked failed. The cancel step tolerated a completed batch but not a failed one. Fold both guards into one helper that tolerates a failed batch only when OpenAI's recorded error is the org's enqueued token limit, and prints the batch's errors so the reason is in the log either way * test: close the injected AsyncOpenAI client after the content policy test * chore(lint): ratchet TQ005 down by the global mutation this branch cleared * chore(lint): ratchet TQ005 to 2660 on the merged tree * chore(lint): ratchet TQ005 to 2561 on the merged tree * chore(lint): ratchet TQ005 to 2548 on the merged tree
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
F811inruff-tests.toml, already wired to CIUser Flow
Before: a developer edits the watsonx credential lookup and the test that checks which secrets it reads can stop compiling its own assertion
WATSONX_API_KEYand send POST https://litellm-domain/v1/chat/completions for a watsonx modelcallthat has quietly replaced thecallhelper the same file importsAfter: the shadowing is rejected before it lands
def test_xthat would silently replace the first, so a test cannot disappear by being renamed onto its neighbourRelevant issues
Linear ticket
Pre-Submission checklist
Screenshots / Proof of Fix
This PR changes lint config and tests, so there is no proxy route to curl. The check that matters for a change made mostly of deletions is that no test went missing and no module stopped importing, so both sides collect the whole suite and the counts are compared.
Before (4b29702)
After (e9d40a8)
Rule and regression numbers
ruff check --config ruff-tests.toml testswent from 344F811findings to zero. Identical collection on both sides: the same 46989 tests, and the same 70 pre-existing collection errors, all of them modules that fail to import for missing credentials or optional dependencies. Nothing new and nothing gone.Type
🧹 Refactoring
🚄 Infrastructure
✅ Test
Caveats (if any)
prisma_clientparameters keep a narrownoqawith the reasontests/load_tests/conftest.pyis new, so pytest shares those fixturesFinal Attestation