test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction

[mubashir1osmani]

Relevant issues

GH #27453, GH #28991, GH #13034

Linear ticket

LIT-3686, LIT-3266, LIT-3221

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

This PR adds tests only; there is no proxy behavior to curl. The concrete signal is that the previously broken collection is now clean and the one non-live regression runs in standard CI

Spend suite collection was aborting the whole e2e session before this change because spend_e2e_client.py imported SpendTagsResponse/TagSpend that were never defined. Collection now succeeds end to end

$ cd tests/e2e && python -m pytest --collect-only -q \
    llm_translation/test_deepseek_reasoning_e2e.py \
    llm_translation/test_chat_completions_regression_e2e.py \
    llm_translation/test_responses_e2e.py \
    llm_translation/test_provider_features_e2e.py \
    batches/test_batches_e2e.py \
    logging/test_prometheus_cardinality_e2e.py \
    spend_tracking/
...
spend_tracking/test_spend_tracking_e2e.py::test_request_tags_round_trip
spend_tracking/test_spend_tracking_e2e.py::test_tag_spend_matches_sum_of_tagged_logs
63 tests collected in 0.12s

The A6 langfuse-eviction regression is a plain unit test (no live proxy) and runs in normal CI

$ python -m pytest tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py -q
x..
2 passed, 1 xfailed in 0.54s

The live e2e cases (A1 through A5) execute in the e2e pipeline against the deployed staging gateway with real provider keys; they skip locally when no proxy answers the liveness probe, per the harness contract

Type

✅ Test

Changes

Closes coverage gaps that the e2e registry flagged as uncovered, plus the collection fix that was hiding the spend suite. Each group is a regression net tied to a specific ticket or issue

• A1 (llm_translation/test_deepseek_reasoning_e2e.py): a control case proves the DeepSeek reasoner returns reasoning_content, and two strict-xfail cases document that reasoning_effort='none' and thinking type='disabled' are silently dropped (LIT-3686 / GH When reasoning_effort is set to none, thinking should be automatically disabled(for deepseek-v4) #27453)
• A2 (llm_translation/test_chat_completions_regression_e2e.py and test_responses_e2e.py): a parametrized net asserting real completion content rather than just a 200 across the configured providers on /chat/completions and /responses (GH [Bug]: Proxy forwards internal _litellm_* reservation fields to OpenAI chat completions #28991)
• A3 (llm_translation/test_provider_features_e2e.py): asserts service_tier is honored and that prompt-cache read tokens grow on a repeated cacheable prefix
• A4 (batches/test_batches_e2e.py): mints a rate-limited key so the batch pre-call rate limiter runs, then asserts the internal input-file retrieval leaves no unattributed spend row behind (LIT-3266)
• A5 (logging/test_prometheus_cardinality_e2e.py): drives one chat per distinct api_key_alias and asserts each alias grows its own labeled litellm_requests_metric_total series on /metrics
• A6 (test_litellm/.../specialty_caches/test_dynamic_logging_cache.py): a strict-xfail regression proving eviction must not close an httpx client still held by an in-flight caller (LIT-3221 / GH [Bug]: Langfuse Reporting Fails with "Cannot send a request, as the client has been closed" #13034)

Supporting changes: tests/e2e/models.py gains the typed request and response fields these tests read (reasoning_effort, thinking, service_tier, key_alias, the cache usage fields, and the spend-log api_key); spend_e2e_client.py gets SpendTagsResponse modeled as a RootModel[list[TagSpend]] so /spend/tags parses as the bare array it actually returns, unblocking the tag-spend tests that had never run

I dropped a stray litellm-regression-tests git submodule that an earlier commit had added; nothing imports it at runtime (only a provenance comment references it), so the gitlink and .gitmodules entry would have made CI init a submodule it never uses

[greptile-apps]

Greptile Summary

This PR is a test-only change that closes e2e coverage gaps across chat/responses translation, provider features (service_tier, prompt caching), batches, Prometheus cardinality, and a unit-level regression for langfuse cache eviction; it also fixes a broken collection path that was preventing the spend suite from running.

• Six new test suites (A1–A5 e2e, A6 unit) plus a strict-xfail documentary test for LIT-3221; the spend client's spend_per_tag access is replaced with the correct .root accessor now that SpendTagsResponse is a RootModel.
• endpoints_client.py is a new shared client for non-chat endpoints; it contains a path bug (/embeddings instead of /v1/embeddings) that would 404 against the gateway for any future test that calls embeddings().
• logging/conftest.py was written for the Prometheus suite but carries a Datadog module docstring and a datadog_creds skip-gate fixture that is neither used nor applicable here.

Confidence Score: 4/5

All changes are test-only; no production code is touched. The bugs found are confined to the new client helper and a conftest, and do not affect the tests that actually run in this PR.

The endpoints_client.py embeddings method will 404 against the gateway due to the missing /v1/ prefix — any test that calls it will fail immediately. None of the new test files in this PR call embeddings(), so it is a latent defect rather than a current test breakage, but it should be fixed before additional callers are added.

tests/e2e/llm_translation/endpoints_client.py (wrong /embeddings path) and tests/e2e/logging/conftest.py (Datadog docstring and dead fixture on a Prometheus suite).

Important Files Changed

Filename	Overview
tests/e2e/llm_translation/endpoints_client.py	New client for non-chat endpoints; embeddings method sends to /embeddings instead of /v1/embeddings, which would 404 against the gateway.
tests/e2e/logging/conftest.py	New conftest for the Prometheus cardinality suite but its docstring and datadog_creds fixture describe a completely different Datadog log-shipping suite; misleading and the fixture is dead code here.
tests/e2e/batches/test_batches_e2e.py	Adds rate-limited-batch test (LIT-3266) and replaces fragile cancel with retry helper; deferred cleanup call omits the provider arg which may silently fail.
tests/e2e/logging/logging_client.py	New Prometheus-aware client for the logging suite; minor indentation inconsistency inside ChatBody constructor call.
tests/e2e/logging/test_prometheus_cardinality_e2e.py	New e2e test polling /metrics to assert per-key-alias labeled series grow correctly; logic and polling pattern are sound.
tests/e2e/llm_translation/test_deepseek_reasoning_e2e.py	New e2e regression tests for DeepSeek reasoning disable; control case plus two strict-xfail cases documenting LIT-3686 correctly.
tests/e2e/models.py	Adds typed fields for reasoning_effort, thinking, service_tier, cache usage, key_alias, and spend tags; SpendTagsResponse correctly modeled as RootModel[list[TagSpend]] to match the bare-array response.
tests/e2e/spend_tracking/spend_e2e_client.py	Switches from data.spend_per_tag to data.root to correctly access the RootModel list; unblocks tag-spend tests that were previously broken at collection time.
tests/test_litellm/litellm_core_utils/specialty_caches/test_dynamic_logging_cache.py	Adds strict-xfail regression documenting LIT-3221 (eviction closing in-flight httpx client); fake objects correctly model the real SDK behavior, and the xfail correctly fires on the current broken eviction path.

_{Reviews (1): Last reviewed commit: "test(e2e): drop unused litellm-regressio..." | Re-trigger Greptile}

[greptile-apps]

The embeddings method sends to /embeddings without the /v1/ prefix, while every other method in this file (responses, messages, rerank, audio_speech, images) correctly uses /v1/…. Any test that calls endpoints_client.embeddings() will receive a 404 from the gateway because the proxy mounts the endpoint under /v1/embeddings.

Suggested change

	def embeddings(self, key: str, model: str, text: str) -> StreamingResponse:
	return self._send("/embeddings", key, EmbeddingsRequest(model=model, input=text))
	def embeddings(self, key: str, model: str, text: str) -> StreamingResponse:
	return self._send("/v1/embeddings", key, EmbeddingsRequest(model=model, input=text))

[mubashir1osmani]

theres 2 variants of embeddings both with and without the prefix v1. tested this locally and it passes

[greptile-apps]

Fair point — the litellm proxy registers the embeddings endpoint at both /embeddings and /v1/embeddings, so the path without the prefix is intentional and valid. My original comment was wrong. Sorry for the noise.

[greptile-apps]

The module docstring and datadog_creds fixture describe a Datadog log-shipping suite (DD_API_KEY, DD_SITE, periodic-flush task), but this conftest backs test_prometheus_cardinality_e2e.py, which scrapes /metrics and checks per-key Prometheus series. The datadog_creds fixture is never requested by the Prometheus test, so the Datadog skip gate will never fire — it's dead code that will confuse anyone adding a Prometheus test.

Suggested change

	"""Fixtures for the Datadog logging suite.
	These tests drive the Datadog batch-send path (#25663) directly against the real
	Datadog logs intake with synthetic events - no LLM calls, no proxy, no log
	read-back - so they need only the shipping credentials DD_API_KEY + DD_SITE
	(DD_SERVICE is an optional tag). No Datadog Application key is required, and they
	skip when the shipping credentials are absent from the environment.
	"""
	"""Fixtures for the logging e2e suite (Prometheus cardinality).
	These tests drive traffic through the gateway with distinct api_key_alias values
	and assert that each alias grows its own labeled series on the proxy's /metrics
	endpoint. No external credentials are required beyond those already set for the
	gateway; the proxy must have prometheus enabled in its callback config.
	"""

[greptile-apps]

The messages and max_tokens keyword arguments inside the ChatBody(...) constructor call are indented at the same level as the ChatBody class name rather than at the same level as model=model. While Python ignores this at runtime, it makes the argument list look like two separate expressions and will confuse any autoformatter.

Suggested change

	ChatBody(
	model=model,
	messages=[ChatMessage(role="user", content=text)],
	max_tokens=64,
	),
	ChatBody(
	model=model,
	messages=[ChatMessage(role="user", content=text)],
	max_tokens=64,
	),

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Missing provider in deferred batch cancel

The cleanup lambda calls client.cancel_batch(batch.id, key=key) without a provider keyword argument. The module-level cancel_batch helper always passes provider to the same method, suggesting BatchClient.cancel_batch requires it. If provider is a required keyword-only parameter, the lambda will raise TypeError at teardown; quietly() will silently swallow it, leaving the batch uncancelled and the gateway with an orphaned resource.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!