chore(release): backport #29015, #29444, #29447, #30480, #30573 to stable/1.88.x and cut 1.88.4

[yuneng-berri]

Relevant issues

Backports four already-merged fixes onto stable/1.88.x and cuts 1.88.4. The picks are #29444 (Datadog batch splitting on 413), #29447 (stop the use_chat_completions_api control flag from leaking into the provider request body), #30480 (cap Anthropic cache_control injection at 4 blocks), and #30573 (AIM guardrail blocks return a conformant 400 instead of a 500). #30573 needs ProxyException to populate Exception.args so str(exc) returns the message; that came from #29015 (LIT-3094), which was not yet on this line, so #29015 is included as a required dependency.

Pre-Submission checklist

• I have added meaningful tests
• My PR's scope is as isolated as possible
• Greptile review requested

What is included

In merge order:

• fix(proxy): populate Exception.args so str(ProxyException) returns message (LIT-3094) #29015 fix(proxy): populate Exception.args so str(ProxyException) returns message (LIT-3094)
• fix(datadog): split oversized batches on 413 instead of re-queueing forever #29444 fix(datadog): split oversized batches on 413 instead of re-queueing forever
• fix: stop use_chat_completions_api flag from leaking into provider request body #29447 fix: stop use_chat_completions_api flag from leaking into provider request body
• fix(integrations): cap Anthropic cache_control injection at 4 blocks #30480 fix(integrations): cap Anthropic cache_control injection at 4 blocks
• fix(guardrails): return 400 not 500 when AIM blocks a request #30573 fix(guardrails): return 400 not 500 when AIM blocks a request
• bump: version 1.88.3 -> 1.88.4
• chore: refresh uv.lock for 1.88.4

#29444, #29447, #30480 are picked verbatim (patch-id identical to their source commits). #30573 and #29015 are adapted; see below.

Adaptation notes

#30573: the upstream change inserts the ProxyException re-raise block right after a call to self._apply_router_cooldown_retry_after(headers, e). That helper does not exist on 1.88.x, so the call line is dropped and only the ProxyException block is applied, attached to the existing error funnel just before the HTTPException branch. The block itself is byte-for-byte the upstream code. The two production changes to proxy/utils.py (excluding ProxyException from llm_exceptions alerting, and classifying it as a proxy-only error) and the aim.py rewrite applied cleanly. The added tests were placed at the equivalent anchors on this line (the upstream insertion points TestStreamCloseOnDisconnect and TestShouldUseSmtpSsl do not exist here), and test_proxy_utils.py needed "patch" added to its module-level unittest.mock import because 1.88.x imported it per-method rather than at module scope. Test bodies are unchanged.

#29015: included as the dependency that makes str(ProxyException) return its message, which #30573's AIM tests assert through pytest.raises(..., match=...). The production change to _types.py (super().init(self.message)) applied cleanly. Its five LIT-3094 regression tests were placed after the existing 1.88.x test in test_proxy_types.py, which is preserved alongside them.

Known noise on this line

The targeted test baseline on stable/1.88.x has one pre-existing failure unrelated to these picks: tests/test_litellm/proxy/test_proxy_utils.py::test_get_custom_url, which asserts a host of 0.0.0.0 while the local environment resolves localhost. It fails identically before and after the picks. Everything else in the targeted set passes (1 failed, 187 passed post-pick; the failure is this known-noise case).

Screenshots / Proof of Fix

Live proxy on stable/1.88.x with the picks applied. The #29447 fix strips the use_chat_completions_api control flag so a strict provider no longer rejects the request:

# use_chat_completions_api is stripped, request succeeds
curl /v1/chat/completions -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"reply with the single word: works"}],"use_chat_completions_api":true}'
-> SUCCESS content: works

# contrast: a genuinely unknown field still leaks and is rejected, proving the flag above was stripped, not silently accepted
curl /v1/chat/completions -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"totally_bogus_unknown_param":true}'
-> REJECTED: litellm.BadRequestError: OpenAIException - Unrecognized request argument supplied: totally_bogus_unknown_param

Proxy sanity green vs the pre-pick baseline: health/liveliness 200, a real gpt-4o-mini completion returns content, and a scoped generated key completes a call. Targeted-test delta is zero new failures. A behavioral stress-test over the picks (symbol resolution, each pick's own tests, and existing callers of every modified function) returned no regressions across all three checks.

Type

🐛 Bug Fix

Changes

Backport-only. No new behavior beyond the cherry-picked fixes and the version bump.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
4 out of 6 committers have signed the CLA.

✅ mateo-berri
✅ ryan-crabbe-berri
✅ shivamrawat1
✅ yuneng-berri
❌ yassin-berriai
❌ oss-agent-shin
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 17.30769% with 86 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...tellm/integrations/anthropic_cache_control_hook.py	16.00%	42 Missing ⚠️
litellm/integrations/datadog/datadog.py	9.52%	38 Missing ⚠️
...itellm/proxy/guardrails/guardrail_hooks/aim/aim.py	42.85%	4 Missing ⚠️
litellm/proxy/common_request_processing.py	33.33%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This backport cherry-picks five targeted bug fixes onto stable/1.88.x and cuts version 1.88.4. Each pick addresses a discrete production defect with a focused code change and dedicated mock-only tests.

• DataDog (fix(datadog): split oversized batches on 413 instead of re-queueing forever #29444): Replaces the infinite 413-requeue loop with _send_with_413_split, which recursively halves oversized batches until they are accepted or reduced to a single undeliverable event (dropped); adds DD_BATCH_SIZE env-var override via _resolve_dd_batch_size().
• Param leak (fix: stop use_chat_completions_api flag from leaking into provider request body #29447): Adds use_chat_completions_api to all_litellm_params so the control flag is stripped from the provider request body rather than forwarded.
• Anthropic cap (fix(integrations): cap Anthropic cache_control injection at 4 blocks #30480): Enforces Anthropic's 4-block cache_control limit in the hook; also fixes a latent bug where role-based injection silently no-oped (local variable reassignment instead of list index write).
• AIM guardrails (fix(guardrails): return 400 not 500 when AIM blocks a request #30573 + fix(proxy): populate Exception.args so str(ProxyException) returns message (LIT-3094) #29015): Replaces bare HTTPException raises with ProxyException via a _rejection() factory, removes the fastapi import from aim.py, excludes guardrail blocks from LLM alerting, and populates Exception.args so str(ProxyException) returns the message.

Confidence Score: 4/5

Safe to merge; all five changes are narrowly scoped bug fixes with thorough mock-only test coverage and no schema or API-contract changes.

The DataDog splitting logic is the most complex change and carries a minor misleading debug log in mock mode (flagged above), but the delivery logic itself is sound and well-tested across six new scenarios including partial-delivery and transient-error cases. The remaining four changes are minimal and each is guarded by dedicated regression tests. The known pre-existing test failure (test_get_custom_url) is documented and unrelated to any of the picks.

litellm/integrations/datadog/datadog.py — the split-on-413 implementation is the most intricate new code; the mock mode success log could be tightened as suggested.

Important Files Changed

Filename	Overview
litellm/integrations/datadog/datadog.py	Replaces the old 413-requeue loop with a _send_with_413_split method that recursively halves oversized batches; adds _resolve_dd_batch_size() to allow env-var override of batch size
litellm/integrations/anthropic_cache_control_hook.py	Refactors cache-control injection to enforce Anthropic's 4-block limit; fixes a latent bug where role-based injection silently dropped its write (local variable reassignment instead of list index assignment)
litellm/proxy/guardrails/guardrail_hooks/aim/aim.py	Replaces HTTPException raises with ProxyException via a new _rejection() factory, removing the fastapi dependency from this file and returning conformant 400 errors on guardrail blocks
litellm/proxy/common_request_processing.py	Inserts a ProxyException branch before the existing HTTPException branch so retry/cooldown headers are forwarded on guardrail rejections
litellm/proxy/_types.py	Adds super().__init__(self.message) to ProxyException.__init__ so str(exc) returns the message instead of an empty string (LIT-3094)
litellm/types/utils.py	Adds use_chat_completions_api to all_litellm_params so the control flag is stripped from the provider request body instead of leaking through
litellm/proxy/utils.py	Excludes ProxyException from LLM alerting and adds it to the "proxy-only error" classification alongside HTTPException

_{Reviews (1): Last reviewed commit: "chore: refresh uv.lock for 1.88.4" | Re-trigger Greptile}

[greptile-apps]

The "successfully mocked" log fires unconditionally after _send_with_413_split, even when undelivered is non-empty. In mock mode with an oversized payload this would print a success message while some events are still queued for re-delivery, making it hard to diagnose partial-delivery scenarios.

Suggested change

	if self.is_mock_mode:
	verbose_logger.debug(
	f"[DATADOG MOCK] Batch of {len(batch_to_send)} events successfully mocked"
	)
	if self.is_mock_mode and not undelivered:
	verbose_logger.debug(
	f"[DATADOG MOCK] Batch of {len(batch_to_send)} events successfully mocked"
	)