Repository navigation
test(e2e): clear the two rc/1.103.0 e2e reds owned by upstream providers - #43388
Conversation
Backports the batch cleanup half of #43321 so an Azure cancel still in progress after 120s leaves a warning instead of a teardown error, and points the MCP Tools UI test at DeepWiki's renamed ask_wiki_question tool.
|
| # cancel is still deferred for cleanup, just not asserted for these two. | ||
| _CANCEL_ASSERTED_PROVIDERS = frozenset({"openai", "bedrock"}) | ||
| # cancel is still deferred for cleanup, just not asserted for Vertex. | ||
| _CANCEL_ASSERTED_PROVIDERS = frozenset({"openai", "azure", "bedrock"}) |
There was a problem hiding this comment.
Azure cancellation flakes return Adding Azure to this set makes every Azure lifecycle case retrieve and cancel the batch during the test. Azure was previously excluded because these provider calls can fail intermittently. If the connection is refused, the helpers do not retry it, so the test fails even when batch creation and retrieval worked and cleanup could proceed.
…43382 and #43388 (#43391) The batch cleanup leftover warning used a class defined in a test-directory module. The xdist controller cannot import it, so an uncaught leftover warning crashed the whole e2e run. It is now a plain UserWarning. The integration harness backport missed the workers option and Gateway.request headers that the OTel conversation id tests use.
TLDR
Problem this solves:
litellm-e2ebuild 308 on rc/1.103.0 went red on two things outside LiteLLM's controltest_batch_lifecycle[azure-unified]errored in teardownask_questiontool toask_wiki_question, so the MCP Tools UI test failedHow it solves it:
cancellingafter 120s, and the file it holds, become warnings instead of errorsask_wiki_questionRelevant issues
Partial backport of #43321
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)tests/e2e/batches/test_batch_cleanup.pypasses on this branch (50 tests). DeepWiki's livetools/listreturnsask_wiki_question,read_wiki_contentsandread_wiki_structureAdaptations from #43321: rc/1.103.0 never had the blank S3 env e2e test, so its removal, the integration test that replaces it, and the coverage registry edits are left out.
batch_cleanup.pyandtest_batch_cleanup.pyare taken from main as of #43321, which also carries a one-line message assertion from #43294Type
✅ Test
Caveats (if any)
Low