test: fix order-dependent flake in passthrough guardrail call-type test - #35317
Merged
mateo-berri merged 1 commit intoJul 31, 2026
Merged
Conversation
… order-dependent flake
Contributor
Greptile SummaryThis PR makes the passthrough guardrail call-type test order-independent.
Confidence Score: 5/5The PR appears safe to merge because the scoped mock targets the actual cached state and restores it after the test. The modified test continues to verify passthrough call-type resolution and handler invocation while removing dependence on whether another test initialized the module-level cache first.
|
| Filename | Overview |
|---|---|
| tests/test_litellm/proxy/pass_through_endpoints/test_passthrough_post_call_guardrails.py | The test now patches the exact cached mapping consumed by the hook, avoiding order-dependent behavior without weakening its assertions. |
Reviews (1): Last reviewed commit: "test: patch unified guardrail mapping gl..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
tin-berri
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
How it solves it:
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
This fixes a unit test that is order-dependent, so the end-to-end surface is the CI test run itself; the failure it eliminates is this proxy-infra job failure on an unrelated PR, where the test failed plus both its reruns on worker gw0 after
test_proxy_logging_hook_detection.pyran first on the same workerDeterministic reproduction before the fix, at commit 8e28765 (the staging base this branch is cut from), using the same xdist mode CI uses. When the polluter module runs first, the flaky test fails exactly like CI:
The leak also fires in the opposite order at 8e28765. Single-process runs sort tests by name, so the flaky test runs first and permanently replaces the mapping global with its mock, which silently disables unified guardrails for every other call type in that worker; three unrelated guardrail-block tests then fail open:
After the fix, at commit 47ebc96, both orderings and the file in isolation pass:
Type
✅ Test
Changes
unified_guardrail.pycachesload_guardrail_translation_mappings()in the module globalendpoint_guardrail_translation_mappings, populated lazily on first hook call and never reset. The flaky test patched the loader function, which is only consulted while the global is stillNone; whenever any earlier test in the same xdist worker had exercised a unified guardrail hook (in the failing CI run, the streaming-block tests intest_proxy_logging_hook_detection.py), the real mappings were already cached, the patch was a no-op, and the real pass-through handler ran instead of the mock, failingassert_awaited_onceon all three attempts since pytest-rerunfailures reruns in the same processThe fix follows the convention already used by
test_blocked_response_usage.pyandtest_unified_guardrail.py: patch the module global itself with the mock mapping viapatch.object, which both makes the test immune to whatever ran before it and restores the previous value afterwards, closing the reverse leak where the test's mock mapping poisoned the global for later tests in the workerFinal Attestation