test(e2e): skip the three Datadog MCP tool-call tests pending LIT-5052

[ryan-crabbe-berri]

TLDR

High level flow for the user:

• Nothing changes for users; test-only
• Three MCP e2e tests stop failing on an upstream schema change

High level flow on a technical level:

• Datadog now rejects the undocumented telemetry argument
• All three tests send it, so every tool call 400s
• Skip them; their coverage cells return to the gap list

Relevant issues

Linear ticket

Refs LIT-5052

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

This PR adds no product behavior, so there is nothing to curl a green result out of; what needs proving is that the upstream tool call is rejected and that the three tests can no longer reach their assertions. Evidence is from the stage e2e run on 2026-07-31 12:47 UTC, both namespaces, at stage image commit 16507f1174

The client-side failure, byte-identical across all three tests and both suites:

E  AssertionError: search_datadog_logs errored: content=[McpCallContent(type='text',
   text='McpError: invalid params: validating "arguments": validating root:
   unexpected additional properties ["telemetry"]')] is_error=True
E  assert True is not True

The gateway logged the upstream rejection at each failure instant (13:33:59, 13:35:15, 13:35:22 UTC):

{"message": "MCP client call_tool failed - Error Type: McpError, Error: invalid params:
 validating \"arguments\": validating root: unexpected additional properties [\"telemetry\"],
 Tool: search_datadog_logs, Server: https://mcp.us5.datadoghq.com/v1/mcp?toolsets=core,
 Transport: MCPTransport.http", "level": "ERROR", "logger": "client.py:606"}

This is not a credential problem, not a Datadog outage, and not a proxy regression. Auth demonstrably works: test_search_logs_finds_seeded_completion failed at line 96, meaning dd_logs.poll_events_for_marker(...) had already returned data over the same DD-API-KEY/DD-APPLICATION-KEY and client.await_tool(...) had already listed search_datadog_logs over the authenticated MCP session. Only tools/call argument validation failed. There is no 401, 403, 429 or 5xx from mcp.us5.datadoghq.com anywhere in the window. And nothing in the proxy injects the key; grep -rn "telemetry" litellm/proxy/_experimental/mcp_server/ matches only two OpenTelemetry doc comments

The argument is hardcoded in the tests themselves, at tests/e2e/mcp/test_mcp_datadog_e2e.py:91, tests/e2e/mcp/test_mcp_guardrail_e2e.py:111 and tests/e2e/mcp/test_mcp_key_access_e2e.py:73. telemetry has never appeared in Datadog's documented parameter list for search_datadog_logs, so the tests were relying on the server silently ignoring unknown properties; Datadog's own docs note the MCP tools are "under significant development and are subject to change"

After this PR, at commit d62e5246de, all three skip with the reason attached:

$ cd tests/e2e && python -m pytest mcp/ -q -rs -k "test_search_logs_finds_seeded_completion or test_content_filter_blocks_banned_keyword_in_tool_args or test_call_tool_denied_without_permission"
sss                                                                      [100%]
SKIPPED [1] mcp/test_mcp_datadog_e2e.py:52: LIT-5052: this test sends a `telemetry` argument ...
SKIPPED [1] mcp/test_mcp_guardrail_e2e.py:81: LIT-5052: the control call sends a `telemetry` argument ...
SKIPPED [1] mcp/test_mcp_key_access_e2e.py:54: LIT-5052: the control call proving a granted key CAN invoke the tool ...
3 skipped, 4 deselected in 0.19s

And the coverage collector hands the cells back rather than counting them covered:

$ cd tests/e2e && PYTHONPATH=. python -m coverage_registry.collector
25 cell(s) are claimed only by skipped tests, so they count as uncovered (unskip the test or drop the marker):
  guardrail.litellm_content_filter.pre_mcp_call.blocks
  ...
  mcp.call_tool.api_key.denied_without_permission
  mcp.call_tool.api_key.succeeds
  mcp.list_tools.api_key.succeeds

make lint-e2e-basedpyright reports 0 errors, 0 warnings, 0 notes

Type

✅ Test

Changes

Adds a pytest.mark.skip to the three MCP tests whose Datadog tool call carries the rejected telemetry argument, each with a reason naming LIT-5052 and describing which assertion is being starved. No product code changes and no other test is touched; test_list_tools_denied_without_permission is deliberately left alone because it never calls the tool and still passes

The @pytest.mark.covers markers and the registry rows are both left in place on purpose. The collector resolves skip state with pytest's own evaluator and counts a cell as covered only when a test that would actually run declares it, so skipping already returns all four cells to the gap list. Deleting the rows instead would shrink the denominator and make three P0 MCP cells disappear from the report rather than show up as the gap they now are

Those four cells, uncovered until the skips lift: mcp.list_tools.api_key.succeeds (P0), mcp.call_tool.api_key.succeeds (P0), mcp.call_tool.api_key.denied_without_permission (P0), guardrail.litellm_content_filter.pre_mcp_call.blocks (P1). The fix in LIT-5052 is to drop the argument, which is small; the skips are meant to be short-lived

QA runbook

• tests/e2e/mcp/test_mcp_datadog_e2e.py::TestDatadogMcpRoundTrip::test_search_logs_finds_seeded_completion - currently skipped; when unskipped it proves a chat completion seeded with a unique marker is findable through the real Datadog MCP server's search tool

  • Asserts the Datadog credentials are present up front and hard-fails rather than skipping, so a missing key turns the run red instead of quietly passing
  • Registers the real Datadog remote MCP server through /v1/mcp/server with the static DD-API-KEY / DD-APPLICATION-KEY headers, deferring its deletion
  • Drives a chat completion carrying a unique marker so there is a specific log line to go find
  • Polls Datadog's Logs Search API for that marker, which proves the log actually landed before the MCP path is blamed for not finding it
  • Waits for search_datadog_logs to appear in the server's tool list, absorbing the registration lag across replicas
  • Calls the tool and asserts the response is not an error and contains the marker; this is the step that currently fails on the telemetry argument
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

• tests/e2e/mcp/test_mcp_guardrail_e2e.py::TestMcpToolCallGuardrail::test_content_filter_blocks_banned_keyword_in_tool_args - currently skipped; when unskipped it proves a content-filter guardrail set to pre_mcp_call blocks a banned keyword inside an MCP tool call's arguments while a clean call still passes

  • Creates a content-filter guardrail configured at the pre_mcp_call hook point with a banned keyword, deferring its deletion
  • Registers the Datadog MCP server and waits for its tool to become callable, anchored to the later of the guardrail write and the server write so neither propagation clock is assumed
  • Issues a tool call whose arguments contain the banned keyword and asserts the guardrail blocks it before it reaches Datadog
  • Issues a clean tool call and asserts it is not blocked, which is the control proving the guardrail discriminates rather than blanket-denying; this control is what currently fails on the telemetry argument
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

• tests/e2e/mcp/test_mcp_key_access_e2e.py::TestMcpKeyWithoutAccessIsDenied::test_call_tool_denied_without_permission - currently skipped; when unskipped it proves a key without MCP permission cannot invoke a tool that a permitted key can

  • Registers the Datadog MCP server and defers its deletion
  • Mints a key granted access to that server and a second key with no MCP access
  • Calls the tool with the granted key and asserts it succeeds, establishing that the tool works and the denial below is about permission rather than a broken call; this control is what currently fails on the telemetry argument
  • Calls the same tool with the unpermitted key and asserts it is denied, which is the contract under test
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR temporarily disables three Datadog-backed MCP end-to-end tests.

• Adds ticket-linked unconditional skips to the Datadog round-trip, MCP guardrail, and tool-call authorization tests.
• Retains their coverage markers so the registry reports the affected cells as uncovered.

Confidence Score: 4/5

The PR appears safe to merge, but unnecessarily leaves four MCP coverage cells unexecuted when removing the rejected test-only argument would preserve the tests.

The coverage registry honestly reports the skipped cells as gaps, but all three tests can retain their intended assertions without the unused telemetry argument, so the unconditional skips create avoidable regression blind spots.

Files Needing Attention: tests/e2e/mcp/test_mcp_datadog_e2e.py, tests/e2e/mcp/test_mcp_guardrail_e2e.py, tests/e2e/mcp/test_mcp_key_access_e2e.py

Important Files Changed

Filename	Overview
tests/e2e/mcp/test_mcp_datadog_e2e.py	Skips the sole end-to-end coverage for API-key-authenticated MCP list-tools and call-tool success paths instead of removing the rejected argument.
tests/e2e/mcp/test_mcp_guardrail_e2e.py	Skips the MCP pre-call content-filter test, leaving its blocked and clean control paths unexecuted.
tests/e2e/mcp/test_mcp_key_access_e2e.py	Skips tool-call permission-denial coverage while leaving the separate list-tools denial test active.

_{Reviews (1): Last reviewed commit: "test(e2e): skip the three Datadog MCP to..." | Re-trigger Greptile}

[greptile-apps]

Skips avoidable regression coverage

These unconditional skips remove the only executing e2e coverage for four MCP success, authorization, and guardrail contracts, even though telemetry is unused by their assertions and can be removed from the three tool calls. The coverage registry reports the resulting gaps honestly, but CI no longer detects regressions in these paths while the skips remain.

Rule Used: What: Flag any modifications to existing tests and... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!