test(lint): ban blind pytest.raises(Exception) with ruff B017

[ryan-crabbe-berri]

TLDR

Problem this solves:

• pytest.raises(Exception) passes on any crash, not the rejection
• A refactor's TypeError satisfies it as well as the real error
• 111 such sites in tests/, several never reaching the code they name

How it solves it:

• Selects ruff B017 in ruff-tests.toml, already wired to CI
• Narrows every site to the type a runtime probe saw it catch
• Adds match= where the code genuinely raises a bare Exception
• Two sites turned out to be failing on setup, never on the behavior

User Flow

Before: a developer cancels a response id the gateway never issued, and nothing checks what comes back

1. They POST https://litellm-domain/v1/responses and get an id back
2. They POST https://litellm-domain/v1/responses/resp_never_issued/cancel with an id that was never issued
3. The gateway answers 404 naming the unknown response id
4. A later change makes that route answer 500 on every cancel, valid id or not
5. CI stays green: the test covering the invalid-id cancel accepts any exception at all, and it had never been reaching a provider in the first place, failing earlier on a missing provider argument

After: the same change turns CI red before a developer ever sees it

1. The same change is made
2. CI runs the same test file and it fails, naming the provider error it expected and the crash it got instead
3. Any new test written to accept any exception is rejected at lint time, so the next one cannot reach the proxy either

Relevant issues

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This PR changes lint config and tests, so there is no proxy route to curl. The proof is three mutations: break a behavior a test exists to police, and show the old test does not notice while the new one does. Every command is identical on both sides; only the tests/ tree differs.

The three mutations, applied one at a time and reverted after each run:

validator_wrong_error  litellm/proxy/_types.py
                       UpdateKeyRequest's validator raises AttributeError instead of
                       ValueError, so pydantic stops turning it into a 422

team_guard_dropped     litellm/proxy/proxy_server.py
                       the "team_id missing from team" guard is deleted, so a malformed
                       default_team_settings entry crashes on the lookup instead

breaker_rewraps        litellm/caching/redis_cache.py
                       the Redis circuit breaker re-raises Exception(str(e)), hiding
                       which failure the caller actually saw

Before (21e9632)

A key-update validator starts failing with the wrong error class

1. Apply validator_wrong_error, then run

pytest tests/proxy_unit_tests/test_proxy_utils.py::test_update_key_request_validation -q

2. Output:

1 passed, 1 warning in 0.06s

A malformed team entry crashes instead of being rejected

1. Apply team_guard_dropped, then run

pytest tests/test_litellm/proxy/proxy_server/test_proxy_config.py::test_ProxyConfig__get_team_config_missing_team_id_raises -q

2. Output:

1 passed, 2 warnings in 2.45s

The Redis breaker hides which failure the caller saw

1. Apply breaker_rewraps, then run

pytest "tests/test_litellm/caching/test_redis_cache.py::test_only_connectivity_failures_open_the_breaker" -q

2. Output:

5 passed, 1 warning in 0.15s

After (ef07d8c)

A key-update validator starts failing with the wrong error class

1. Apply validator_wrong_error, then run

pytest tests/proxy_unit_tests/test_proxy_utils.py::test_update_key_request_validation -q

2. Output:

FAILED tests/proxy_unit_tests/test_proxy_utils.py::test_update_key_request_validation
1 failed, 1 warning in 0.15s

A malformed team entry crashes instead of being rejected

1. Apply team_guard_dropped, then run

pytest tests/test_litellm/proxy/proxy_server/test_proxy_config.py::test_ProxyConfig__get_team_config_missing_team_id_raises -q

2. Output:

FAILED tests/test_litellm/proxy/proxy_server/test_proxy_config.py::test_ProxyConfig__get_team_config_missing_team_id_raises
1 failed, 2 warnings in 1.73s

The Redis breaker hides which failure the caller saw

1. Apply breaker_rewraps, then run

pytest "tests/test_litellm/caching/test_redis_cache.py::test_only_connectivity_failures_open_the_breaker" -q

2. Output:

FAILED ...test_only_connectivity_failures_open_the_breaker[timeout_is_unhealthy]
FAILED ...test_only_connectivity_failures_open_the_breaker[wrong_type_command_is_not]
5 failed, 1 warning in 0.26s

Rule and regression numbers

ruff check --config ruff-tests.toml tests went from 111 B017 findings to zero. Ruff has no autofix for this rule, so every site was edited by hand or by a script driven off the probe.

The 55 touched test files were run in full at both hashes. 34 tests fail identically on each side, all of them live-provider tests with no Gemini, Bedrock, Azure or Databricks credentials on this machine. Exactly one test differs:

NEW at ef07d8c63c: tests/guardrails_tests/test_bedrock_guardrails.py::test_bedrock_guardrails_with_streaming

That is the finding, not a regression. 14 of the 15 tests in that file already fail here without AWS credentials; this one passed only because the NoCredentialsError boto3 raised, long before the guardrail ran, satisfied pytest.raises(Exception). With credentials present it exercises the guardrail block it was written for.

Type

🧹 Refactoring
🚄 Infrastructure
✅ Test

Caveats (if any)

• One site keeps Exception under a noqa: the code raises Exception() with no message
• Nine live-API sites were narrowed by reading the code, not by probing
• Two spend-flush tests raise TypeError from their mocks, never the DB error they name
• The shared responses-API cancel test never reaches a provider; follow-up, not fixed here
• Sites whose type depends on a provider key are pinned at the SDK base error

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR enables Ruff B017 for tests and narrows broad pytest.raises(Exception) assertions to specific exception types or message matches.

• Adds B017 to the test lint configuration.
• Tightens exception assertions across guardrail, proxy, provider, caching, and SDK tests.
• Refines the Redis circuit-breaker test to distinguish underlying failures from the open-breaker state.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
ruff-tests.toml	Enables B017 in the existing test lint configuration to reject blind broad-exception assertions.
tests/test_litellm/caching/test_redis_cache.py	Narrows Redis failure assertions and separately validates the transition to an open circuit breaker.
tests/proxy_unit_tests/test_proxy_utils.py	Narrows validation assertions and records the currently observed mock-induced TypeError behavior.
tests/guardrails_tests/test_eu_ai_act_article5.py	Narrows guardrail rejection assertions to HTTPException.
tests/guardrails_tests/test_semantic_guard.py	Narrows semantic guard rejection assertions to HTTPException.

_{Reviews (2): Last reviewed commit: "test(lint): ban blind pytest.raises(Exce..." | Re-trigger Greptile}

[greptile-apps]

Proxy dependency imported outside proxy

These guardrail tests now import FastAPI directly outside proxy/, coupling SDK test collection to a proxy-only dependency. The pattern also occurs in three sibling files.

Rule Used: What: Do not allow fastapi imports on files outsid... (source)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[ryan-crabbe-berri]

The fastapi import isn't a boundary violation: test_lakera_v2.py, test_zscaler_ai_guard.py and test_lasso_guardrails.py already import HTTPException the same way, and no lint gate forbids it.

[ryan-crabbe-berri]

@greptile re review