fix(datadog): chunk on 413 and honor DD_BATCH_SIZE (LIT-3407)

[oss-agent-shin]

What

Datadog 413 (Payload Too Large) handling is broken end-to-end on
litellm_oss_agent_shin_daily_branch:

async_send_compressed_data runs through the LiteLLM async httpx handler,
which converts any 4xx into a MaskedHTTPStatusError via
_raise_masked_async_error before returning. The existing
if response.status_code == 413 branch inside async_send_batch is therefore
unreachable - the bare except catches the masked exception, re-queues the
entire oversized batch unchanged, and the same payload is replayed on every
flush interval. No logs are ever delivered; pod logs are also spammed
indefinitely by repeated 413s. Even if the branch were reachable, the handler
just re-queued without chunking - Datadog's API enforces a 5 MB uncompressed
limit per call and a payload that hit it once will hit it forever.

Fix

litellm/integrations/datadog/datadog.py:

• Refactor async_send_batch into a small driver + _send_batch_with_413_split
  helper.
• New helper recognises a 413 on both the exception path
  (MaskedHTTPStatusError(.response.status_code == 413) - the actual production
  failure mode) and a defensively-handled returned-413 Response (sync clients,
  mocks, future refactors).
• On a 413 with len(batch) > 1: split the batch in half and recurse on each
  half independently. A successful half is not re-queued by a failing half
  (the previous bare-except behaviour would duplicate successful events on a
  partial failure).
• On a 413 with len(batch) == 1: drop the single event with a verbose error
  log. Re-queueing a single oversized event forever was the root of the
  report's "logs spammed indefinitely" symptom.
• Non-413 transport errors keep the prior contract: re-queue the slice for the
  next flush attempt.

litellm/types/integrations/datadog.py:

• DD_ERRORS.DATADOG_413_ERROR previously pointed operators at DD_BATCH_SIZE
  as the documented workaround, but DD_BATCH_SIZE was never read from the
  environment anywhere in the codebase. The message is updated to surface the
  actually-supported escape hatches
  (litellm.turn_off_message_logging = True /
  DatadogInitParams(turn_off_message_logging=True)).

Tests

tests/test_litellm/integrations/datadog/test_datadog_413_chunking.py (new):
masked-exception 413 path, partial-split success (failing half doesn't
penalise the working half), defensive returned-413, non-413 still re-queues,
and the _is_413_error helper itself.

tests/test_litellm/integrations/datadog/test_datadog_logger_batching.py:
the legacy test_async_send_batch_requeues_events_on_413 is updated and
renamed to test_async_send_batch_splits_and_drops_on_413_response; it now
asserts the correct behaviour (3 send attempts, queue drained) instead of
the broken "requeue forever" contract.

$ python3 -m pytest tests/test_litellm/integrations/datadog/test_datadog_413_chunking.py \
    tests/test_litellm/integrations/datadog/test_datadog_logger_batching.py -q
17 passed in 3.44s

Evidence

Real DataDogLogger.async_send_batch() driven with a mock
async_send_compressed_data that mirrors production
(MaskedHTTPStatusError(413)):

BEFORE (pre-LIT-3407)
============================================================
  flush #1: queue_len=10 send_calls=1
  flush #2: queue_len=10 send_calls=2
  flush #3: queue_len=10 send_calls=3
  >> queue size stays at 10 across every flush; no chunking; same 413 fires
     forever; nothing is ever delivered.

AFTER (this PR)
============================================================
  flush #1: queue_len=0 send_calls=19
  flush #2: queue_len=0 send_calls=19
  flush #3: queue_len=0 send_calls=19
  >> 10 events recursively halved (full binary tree with 10 leaves -> 19
     nodes / send attempts), each single event dropped with a verbose error
     log; queue drains cleanly so the next flush is a no-op.

Partial split — first half rejected (413), second half accepted (202):

AFTER (partial success: first half 413, second half 202)
============================================================
  total send calls=5
  queue after flush=[]
  delivered batches=[['{"event": 2}', '{"event": 3}']]
  >> first half (events 0,1) was chunked down and dropped, second half
     (events 2,3) was delivered in a single call. Working half is not
     penalised by the failing half.

Scope

• DD_BATCH_SIZE env-var support (the phantom workaround in the original
  error string) is intentionally out of scope. Wiring it requires a
  coordinated BerriAI/litellm-docs change (adding DD_BATCH_SIZE to
  docs/my-website/docs/proxy/config_settings.md, which the
  documentation_test_env_keys CI gate validates). That's a separate PR;
  this one is the headline 413-loop fix and is shippable independently.
• This PR removes the misleading DD_BATCH_SIZE reference from the error
  string so we don't ship a hint we don't honour.

Notes

• Pushed via the PUT /repos/.../contents/{path} API because the current
  GITHUB_TOKEN lacks repo scope for git push; the merge-base diff is one
  commit per file but the net change is 4 files.
• No request/response body is read or logged by the fix; this PR cannot leak
  secrets.

Resolves LIT-3407.

Verification (ship-pr)

Check	Result
Base branch	litellm_oss_agent_shin_daily_branch (fork-PR policy) ✅
Customer name leaked	None — PR title and body reference only LIT-3407 and technical context, no project name ✅
Secrets in diff	None — no API keys, tokens, or PII in code or evidence ✅
GitHub Actions	44/44 green, 0 failures (code-quality, documentation, lint, secret-scan, semgrep, all Run tests jobs) ✅
Veria AI - PR Review	✅ success
Greptile review score	4/5 ("safe to merge — the two flagged issues are non-blocking logging and ordering concerns"); both addressed in 11a5e57 ✅
Mergeable state	clean ✅
Unit tests	17 passed in 3.44s; full tests/test_litellm/integrations/datadog/ previously: 61 passed ✅
Runtime evidence	Inline in ### Evidence: real DataDogLogger.async_send_batch() driven with mocked MaskedHTTPStatusError(413) — before flushes leave queue at 10 forever; after flush #1 drains to 0 and stays drained. Partial-success scenario delivers the working half while dropping the failing single events ✅
Codecov	Patch coverage 87.71% (above repo bar) ✅

Ready for human merge.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

Fixes a Datadog 413 infinite-retry loop where async_send_compressed_data raises MaskedHTTPStatusError (not a plain Response), making the old if response.status_code == 413 branch unreachable and causing every oversized batch to be re-queued forever.

• Introduces _is_413_error() to detect a 413 inside a MaskedHTTPStatusError, and refactors async_send_batch into a _send_batch_with_413_split / _handle_413_split recursive pair that halves the batch on 413 until single events are reached (and drops them to break the loop).
• Updates DD_ERRORS.DATADOG_413_ERROR to remove the unsupported DD_BATCH_SIZE hint and point operators at turn_off_message_logging instead.
• Both previous review findings (use verbose_logger.error for the single-event drop outside an except context; prepend failed slices instead of append) are correctly implemented in this revision.

Confidence Score: 5/5

Safe to merge — the change is narrowly scoped to the Datadog batch-send path, all prior review feedback has been applied, and every code path is covered by mock-only unit tests that pass.

The recursive split logic is sound: _is_413_error correctly reaches the status code through MaskedHTTPStatusError.response, the two new helpers never swallow exceptions silently, and the single-event terminal case breaks the loop with verbose_logger.error outside any except context. The updated legacy test still exercises the 413 scenario with the same mock; only the assertions were corrected to match the new behavior, so test coverage is strengthened rather than weakened.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/integrations/datadog/datadog.py	Core fix: refactors async_send_batch into a recursive split helper that correctly handles 413 on both the exception path (MaskedHTTPStatusError) and defensive returned-Response path; prepend ordering for non-413 requeue and verbose_logger.error (not .exception) for single-event drop are both correctly implemented per prior review feedback
litellm/types/integrations/datadog.py	DATADOG_413_ERROR message updated to remove the unsupported DD_BATCH_SIZE hint and document the actually-supported escape hatches (turn_off_message_logging)
tests/test_litellm/integrations/datadog/test_datadog_413_chunking.py	New regression test file covering the masked-exception 413 path, partial-split success, defensive returned-413, non-413 requeue, and the _is_413_error helper; all mock-only with no network calls
tests/test_litellm/integrations/datadog/test_datadog_logger_batching.py	Legacy test updated: mock still returns 413 unchanged, but assertions now verify the correct split-and-drop behavior instead of the old infinite-requeue contract; strengthens rather than weakens coverage

_{Reviews (2): Last reviewed commit: "fix(datadog): Greptile feedback - .error..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 86.36364% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/datadog/datadog.py	86.04%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!

[oss-agent-shin]

@greptileai review

Addressed your two non-blocking findings (commit 11a5e57):

1. _handle_413_split single-event drop now uses verbose_logger.error instead of .exception — the call site isn't inside a try/except, so .exception was producing a "NoneType: None" traceback for non-existent state.
2. Non-413 transport-error requeue path is back to prepending the failed slice (self.log_queue = batch_to_send + self.log_queue) instead of appending — matches the legacy ordering you flagged.

Tests still pass (17 passed). No new code paths.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.