test(e2e): source the cheap Anthropic model from e2e_config across suites - #32964
test(e2e): source the cheap Anthropic model from e2e_config across suites#32964mateo-berri wants to merge 4 commits into
Conversation
Greptile SummaryPure test-code refactor that replaces 22 hardcoded
Confidence Score: 5/5Safe to merge — test-only refactor with no proxy code touched and verified clean against a live stack. Every change is a mechanical substitution of a string literal with a shared constant; no test logic, assertions, or fixtures were weakened. The provider-route derivation in conftest.py was tightened to use the same constant, which is a strict improvement. All touched modules were verified with basedpyright (0 errors) and the coverage registry collector, and the affected test functions were run against a real Anthropic-backed proxy. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/e2e/e2e_config.py | Defines CHEAP_ANTHROPIC_MODEL constant (env-backed, default "claude-haiku-4-5") that all changed files now import — the source of truth for this refactor |
| tests/e2e/quota_management/spend_tracking/conftest.py | DRIVER_MODELS now derives the provider route dynamically as f"anthropic/{CHEAP_ANTHROPIC_MODEL}", addressing the previous review concern about alias/route mismatch |
| tests/e2e/quota_management/spend_tracking/test_spend_tracking_e2e.py | Row-matching assertions updated to use CHEAP_ANTHROPIC_MODEL; substring matching is correct since the logged proxy alias will equal the constant value |
| tests/e2e/llm_translation/test_passthrough_e2e.py | Three call sites updated to CHEAP_ANTHROPIC_MODEL; docstring updated from the literal model name to the constant name |
| tests/e2e/llm_translation/test_chat_completions_regression_e2e.py | CHAT_MODELS tuple anthropic entry updated to CHEAP_ANTHROPIC_MODEL — mechanical substitution, no logic change |
Reviews (2): Last reviewed commit: "fix(e2e): derive the anthropic driver ro..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
This PR is stacked on #32914, which introduces the
CHEAP_ANTHROPIC_MODELconstant in tests/e2e/e2e_config.py; merge that firstThis is a pure test-code refactor with no proxy behavior change, so the proof is the touched suites still passing against a live proxy plus the type and registry gates, all captured at commit e44aa07 against the tests/e2e docker compose stack (ghcr.io/berriai/litellm:main-latest with the suite's inline config, mapped to host port 4010 here because 4000 was taken by another local proxy) with real Anthropic and OpenAI traffic:
The
--strictcollector run performs a full pytest collect-only pass over tests/e2e, so it proves every touched module still imports and everycoversmarker still resolves to a registry cellType
🧹 Refactoring
Changes
Follow-up to #32914, which added
CHEAP_ANTHROPIC_MODELto tests/e2e/e2e_config.py (envE2E_CHEAP_ANTHROPIC_MODEL, default claude-haiku-4-5) after review feedback that a hardcoded model name is hard to maintain, but only used it in the new ratelimit suite; the older suites still hardcode the alias, so retiring or swapping the cheap Anthropic model would mean touching dozens of call sitesThis replaces the 22 remaining hardcoded
"claude-haiku-4-5"proxy-alias literals across 16 files with the shared constant, each with the matching e2e_config import: 14 occurrences across 12 files in quota_management/budgets/, the DRIVER_MODELS proxy alias in quota_management/spend_tracking/conftest.py plus a call and two row-matching assertions in test_spend_tracking_e2e.py, the 3 Anthropic passthrough calls in llm_translation/test_passthrough_e2e.py, and the claude row of CHAT_MODELS in llm_translation/test_chat_completions_regression_e2e.py. Every occurrence was a standalone literal, so no f-strings were needed and no behavior changesFollowing review feedback (commit 65e90c2, gates rerun clean: basedpyright 0 errors,
collector --strictpasses, the spend_calculate test passes live), the spend_tracking DRIVER_MODELS row now derives its provider route asf"anthropic/{CHEAP_ANTHROPIC_MODEL}", so overriding the env var on a proxy where the alias is not pre-registered can no longer silently register the new alias against the old upstream model, and the passthrough module docstring references CHEAP_ANTHROPIC_MODEL instead of naming the modelDeliberately left alone: the upstream provider route id
"anthropic/claude-haiku-4-5"in llm_translation/test_messages_e2e.py (that test registers its owne2e-messages-*alias, so its alias and route stay consistent regardless of the env var), dated bedrock model ids, and the docker-compose.yml inline config, which defines the alias the stack actually serves and matches the constant's default