litellm_fix_mapped_tests_core: disable aiohttp transport in tests#20194
litellm_fix_mapped_tests_core: disable aiohttp transport in tests#20194shin-bot-litellm wants to merge 1 commit intomainfrom
Conversation
## Problem Tests using @respx.mock were hitting real APIs because respx only intercepts httpx requests, but litellm uses aiohttp transport by default. Tests affected: - test_send_email_missing_api_key - test_send_email_multiple_recipients (resend & sendgrid) - test_search_uses_registry_credentials - test_vector_store_create_with_simple_provider_name - test_image_edit_merges_headers_and_extra_headers ## Solution 1. Add disable_aiohttp_transport fixture (autouse=True) to test files that use respx mocking. This forces httpx transport so respx works. 2. Clear client cache in fixture to prevent reuse of old clients. 3. Fix isinstance checks in vector store tests to use type name comparison (avoids module identity issues from sys.path.insert). ## Regression PR #19829 (commit f95572e) added @respx.mock but didn't account for aiohttp transport bypassing respx interception.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
Greptile OverviewGreptile SummaryChanges OverviewThis PR fixes a test reliability issue where tests using What Changed
Technical AnalysisThe fix is well-designed and addresses the regression from PR #19829. By forcing httpx transport, respx can now properly intercept HTTP calls in these test files:
The fixture implementation follows best practices by:
Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| tests/test_litellm/enterprise/enterprise_callbacks/send_emails/test_resend_email.py | Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking |
| tests/test_litellm/enterprise/enterprise_callbacks/send_emails/test_sendgrid_email.py | Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking |
| tests/test_litellm/test_main.py | Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking |
| tests/test_litellm/vector_stores/test_vector_store_create_provider_logic.py | Replaces isinstance checks with type name comparison to avoid module identity issues from sys.path.insert |
| tests/test_litellm/vector_stores/test_vector_store_registry.py | Adds disable_aiohttp_transport fixture to force httpx transport for respx mocking |
Sequence Diagram
sequenceDiagram
participant Test as Test Execution
participant Fixture as disable_aiohttp_transport
participant LiteLLM as litellm module
participant Cache as in_memory_llm_clients_cache
participant Respx as @respx.mock
participant HTTPX as httpx transport
Note over Test,HTTPX: Before PR (Broken)
Test->>Respx: Start test with @respx.mock
Respx->>LiteLLM: Intercept HTTP calls
LiteLLM->>LiteLLM: Use aiohttp transport (default)
Note over Respx,LiteLLM: respx cannot intercept aiohttp!
LiteLLM-->>Test: Real API call made ❌
Note over Test,HTTPX: After PR (Fixed)
Test->>Fixture: autouse=True fixture runs
Fixture->>LiteLLM: Set disable_aiohttp_transport = True
Fixture->>Cache: flush_cache() to clear old clients
Cache-->>Fixture: Cache cleared
Fixture->>Test: Setup complete
Test->>Respx: Start test with @respx.mock
Respx->>LiteLLM: Intercept HTTP calls
LiteLLM->>HTTPX: Use httpx transport (forced)
Respx->>HTTPX: Mock HTTP request
HTTPX-->>Respx: Return mocked response
Respx-->>Test: Test passes with mock ✓
Test->>Fixture: Test complete, teardown
Fixture->>LiteLLM: Restore original disable_aiohttp_transport
Fixture-->>Test: Cleanup complete
Problem
Tests using
@respx.mockwere hitting real APIs because respx only intercepts httpx requests, but litellm uses aiohttp transport by default.Tests affected:
test_send_email_missing_api_keytest_send_email_multiple_recipients(resend & sendgrid)test_search_uses_registry_credentialstest_vector_store_create_with_simple_provider_nametest_image_edit_merges_headers_and_extra_headersSolution
disable_aiohttp_transportfixture (autouse=True) to test files that use respx mocking. This forces httpx transport so respx can intercept requests.isinstancechecks in vector store tests to use type name comparison (avoids module identity issues fromsys.path.insert).Regression
PR #19829 (commit
f95572e3ed) added@respx.mockbut did not account for aiohttp transport bypassing respx interception.Testing
disable_aiohttp_transportfixture ensures respx can mock all HTTP calls