Skip to content

test: add /v1/messages to supported_endpoints schema enum - #32739

Merged
yucheng-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_supported_endpoints_v1_messages
Jul 10, 2026
Merged

test: add /v1/messages to supported_endpoints schema enum#32739
yucheng-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_supported_endpoints_v1_messages

Conversation

@tin-berri

@tin-berri tin-berri commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

CI is currently red on every backend PR against litellm_internal_staging; #32727 shows the same failure on an unrelated change

  • feat: add Meta Model API provider and muse-spark-1.1 (day-0) #32701 added the meta/muse-spark-1.1 entry to model_prices_and_context_window.json with "/v1/messages" in its supported_endpoints
  • test_aaamodel_prices_and_context_window_json_is_valid validates every entry of that json against a hardcoded endpoint allow-list inside tests/test_litellm/test_utils.py; the list predates /v1/messages, so jsonschema rejects the new entry
  • the misc / Run tests job runs against the merge of the PR branch with current staging, so the muse-spark entry is present in every backend PR's run and the job fails on the same assertion regardless of what the PR itself changes
  • adding "/v1/messages" to the allow-list makes the muse-spark entry validate again; the list has no consumers outside this one test, so the one-line change fixes the job for every PR without touching product behavior

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

The defect is inside the test suite itself (a stale schema allow-list), so there is no runtime surface to curl; the proof is the exact check CI runs, captured at the pre-fix and post-fix commits

Before, at 2a12707372 (current litellm_internal_staging tip, which contains the muse-spark entry from #32701):

git checkout 2a12707372 -- tests/test_litellm/test_utils.py
python -m pytest "tests/test_litellm/test_utils.py::test_aaamodel_prices_and_context_window_json_is_valid" -q
E           jsonschema.exceptions.ValidationError: '/v1/messages' is not one of ['/v1/responses', '/v1/embeddings', '/v1/chat/completions', '/v1/completions', '/v1/images/generations', '/v1/realtime', '/v1/realtime/transcription_sessions', '/v1/images/variations', '/v1/images/edits', '/v1/batch', '/v1/audio/transcriptions', '/v1/audio/speech', '/v1/ocr', '/vertex_ai/live', '/v1/realtime/transcription_sessions']
E           On instance['meta/muse-spark-1.1']['supported_endpoints'][2]:
E               '/v1/messages'
1 failed in 0.51s

After, at 5409045644 (this PR):

python -m pytest "tests/test_litellm/test_utils.py::test_aaamodel_prices_and_context_window_json_is_valid" -q
1 passed in 0.36s

The full file also passes at 5409045644: python -m pytest tests/test_litellm/test_utils.py -q gives 227 passed in 0.66s

Type

✅ Test

Changes

#32701 added the meta/muse-spark-1.1 entry to model_prices_and_context_window.json declaring /v1/messages in supported_endpoints, without extending the endpoint allow-list that test_aaamodel_prices_and_context_window_json_is_valid validates the json against. The test therefore fails on litellm_internal_staging itself, and since CI runs against the merge of each PR branch with current staging, every backend PR fails this job regardless of its own diff

Extending the enum (rather than editing the muse-spark entry) is the behavior-preserving fix. /v1/messages is a real served route (@router.post("/v1/messages") in litellm/proxy/anthropic_endpoints/endpoints.py) and the value is load-bearing at runtime: _deployment_passes_through_anthropic_messages in litellm/llms/anthropic/experimental_pass_through/messages/handler.py gates Anthropic-spec passthrough on exactly this entry, so stripping it from the model entry would disable the feature #32701 shipped. The test enum has no consumers outside this test, so adding the member changes no product behavior

The same enum listed /v1/realtime/transcription_sessions twice; this PR drops the duplicate. Net diff is one line added, one removed

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a CI-breaking test failure caused by a missing enum entry in the JSON schema validator used by test_aaamodel_prices_and_context_window_json_is_valid. The prior PR (#32701) added meta/muse-spark-1.1 with /v1/messages in supported_endpoints, but the allow-list in this test was never updated to include it, causing every subsequent backend PR to fail CI.

  • Adds /v1/messages to the supported_endpoints enum in the schema validator — /v1/messages is a real served route and a load-bearing value for Anthropic passthrough logic, so expanding the enum is the correct fix rather than removing it from the model entry.
  • Removes a duplicate /v1/realtime/transcription_sessions entry that was already present in the list.

Confidence Score: 5/5

Safe to merge — the change is a minimal, targeted test fix with no production code touched.

The only change is adding a real, existing endpoint to a validation enum and removing a duplicate entry. It correctly unblocks CI by bringing the test allow-list in sync with the model data, and does not weaken any assertion or mask a regression.

No files require special attention.

Important Files Changed

Filename Overview
tests/test_litellm/test_utils.py Extends the supported_endpoints schema enum to include /v1/messages (a real served route) and removes a duplicate /v1/realtime/transcription_sessions entry — unblocks CI without weakening validation

Reviews (1): Last reviewed commit: "test: add /v1/messages to supported_endp..." | Re-trigger Greptile

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_supported_endpoints_v1_messages (5409045) with litellm_internal_staging (2a12707)

Open in CodSpeed

@yucheng-berri
yucheng-berri merged commit bf02a4a into litellm_internal_staging Jul 10, 2026
128 checks passed
@yucheng-berri
yucheng-berri deleted the litellm_supported_endpoints_v1_messages branch July 10, 2026 07:23
yuneng-berri added a commit that referenced this pull request Jul 11, 2026
…1.92.0 stable cut (#32959)

* fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056) (#32389)

* fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056)

* test(register_model): use a triple provider prefix as the unresolvable-key fixture

get_model_info now resolves bedrock/bedrock/... like a routing prefix, so the
double-prefix fixture stopped exercising the register_model fallback path.
Lock the new double-prefix resolution in as a model-info regression test

(cherry picked from commit 734fd29)

* fix(guardrails): walk Responses-API text taxonomy in shared content helpers (#32542)

* fix(guardrails): walk Responses-API text taxonomy in shared content helpers

Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently
drops all text on the /v1/responses path. AIM turns it into a loud 422 (
{"error":"No messages in the request"}); every other guardrail (Lakera v2,
Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret
detection) scans an empty payload and lets the request through unscanned.

Three defects, all in _content_utils.py:

1. _iter_text_parts_in_content recognised only part.type == "text", but the
   Responses API uses input_text (request) and output_text (assistant).
2. _coerce_input_to_messages gated on "every item has a role key"; any
   Responses input list containing a function_call or function_call_output
   item failed the check and was wrapped as one opaque blob.
3. build_inspection_messages forwarded any role through, including a bare
   tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze
   reject with a schema error.

Fix walks the actual Responses item taxonomy (message, function_call,
function_call_output, bare content parts and strings), recognises
{text, input_text, output_text} everywhere, and coerces any role outside
{system, user, assistant} to user in the outbound inspection payload.

* style: ruff-format changed guardrail files

* test(guardrails): cover function_call_output string form; drop em-dash in new docstring

* fix(guardrails): map function_call_output straight to user role

Avoids ever materialising a schema-invalid bare tool message. The
downstream role-safety coercion in build_inspection_messages still
guards genuinely caller-supplied non-standard roles (developer,
function, custom values); add a regression test covering that path
so the coercion has real coverage after this simplification.

* test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages

* docs(test): soften AIM-specific claims in LIT-4294 test docstrings

Ryan's review flagged that several test docstrings assert AIM's
/fw/v1/analyze validates + rejects specific schema violations. That
behavior is customer-reported in the LIT-4294 writeup, not directly
verified by us. Rephrase to attribute the AIM 422 to the customer's
writeup and describe the underlying constraint as the OpenAI chat
schema; any downstream API that validates against that schema rejects
the same shape.

* refactor(guardrails): move unsupported-role coercion into AIM only

The generic coercion in build_inspection_messages collapsed any role
outside {system, user, assistant} to user for every caller of the
helper. Combined with the pre-existing apply_redacted_messages_back
write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400
on chat-completions tool-message masking into a silent semantic
corruption of the outbound request (role tool with tool_call_id got
rewritten to bare role user, dropping the assistant + tool_calls
sibling).

AIM specifically requires the coercion because its /fw/v1/analyze
validates the payload against the OpenAI chat schema; other guardrails
either do not validate roles or do their own reconstruction. Move the
coercion to AimGuardrail._build_aim_inspection_messages so the shared
helper keeps caller roles intact and no new cross-guardrail role
corruption is introduced. The pre-existing apply_redacted_messages_back
structural flatten remains as separate follow-up work.

function_call_output items still synthesise role user in the shared
helper because they have no natural role field, which is a different
concern from coercing a caller-supplied role.

* refactor(guardrails): preserve role fidelity in shared _content_utils

Shared inspection helpers should extract text and preserve semantic
role signals; role coercion for third-party schema safety stays inside
the guardrail that needs it (AIM).

Three shared-helper changes:
- Bare content-part dicts (input_text/output_text) with an explicit role
  keep it; only role-less parts default to user.
- Responses message items already had their role preserved; the
  behavior is now covered by an explicit test.
- function_call_output items default to role tool (semantic equivalent
  of the chat-completions tool message shape) instead of role user, so
  Responses and chat completions produce symmetric inspection payloads.
  A caller-supplied role on the item is still preserved.

AIM's schema-safe coercion in _build_aim_inspection_messages already
handles the resulting role tool: it collapses to user before the POST
to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the
bare tool message (no tool_call_id can survive the flatten). Added a
regression test in test_aim.py covering that path.

(cherry picked from commit e84a19a)

* feat: add Meta Model API provider and muse-spark-1.1 (day-0) (#32701)

(cherry picked from commit d82645d)

* fix(bedrock): keep mid-conversation system messages in place for Claude Invoke (#32578)

Hoisting every role system entry into the top-level system field mutates
the cache prefix whenever a client such as Claude Code appends a new
mid-conversation system message, invalidating the prompt cache for the
entire message history on Bedrock Invoke. Bedrock only rejects a system
entry at messages.0, so hoist just the leading run and forward the rest
in place

(cherry picked from commit cc36d54)

* feat(otel): emit the gen_ai.client.operation.exception event on failed LLM calls (#32655)

* feat(otel): emit the gen_ai.client.operation.exception event on failed LLM calls

The GenAI semantic conventions record failures of a GenAI client operation as
a log-based event named gen_ai.client.operation.exception, carrying the
exception.type / exception.message / exception.stacktrace trio at severity
WARN and correlated to the failed span. OTel v2 never emitted it: a failed LLM
call produced only the deprecated error.* span attributes, a generic exception
span event without a stacktrace, and the stacktrace under the vendor key
litellm.provider.error.stack_trace.

Build the logs pipeline (LoggerProvider + console/OTLP log exporters mirroring
the metrics plumbing) and record the event behind the enable_events flag, which
until now was defined but consumed nowhere. An operator-configured LoggerProvider
global is reused so the events ride their existing logs pipeline; an explicit
NoOpLoggerProvider global is honored as an opt-out and builds no recorder at all.

The existing span-side error surface (error.type, error.message, the exception
span event, and the litellm.provider.error.* detail keys) is untouched for
backwards compatibility.

* fix(otel): always ride the semconv-required exception pair on the GenAI event

Filtering the event attributes on truthiness conflated "absent" with "empty",
so an empty exception.type or exception.message would have been dropped, leaving
an event with neither semconv-required field. Build the attributes so the pair is
unconditional and only the recommended stacktrace is omitted when the payload
carries none.

* docs(otel): document the events plumbing module in the package README

* test(otel): cover the log exporter selection and logs endpoint normalization

The new logs plumbing had no coverage for exporter-kind selection, the
console fallback for an unrecognized kind, the /v1/logs signal-path rewriting
that lets one OTEL_ENDPOINT serve every signal, or the simple-vs-batch
processor split.

(cherry picked from commit 99b4c5e)

* fix(bedrock): gate in-place system role messages on model support for Claude Invoke (#32831)

* fix(bedrock): gate in-place system role messages on model support for Claude Invoke

* feat(bedrock): default unmapped Claude 4.8+ to in-place system role handling via fallback rule

(cherry picked from commit 5e23a5a)

* fix(anthropic): translate adaptive thinking/effort to pre-4.6 model support (#32867)

* fix(anthropic): translate adaptive thinking/effort to pre-4.6 model support

AnthropicMessagesConfig now reshapes the 4.6+ adaptive-thinking interface
(thinking:{type:adaptive} + output_config:{effort:...}) to whatever the routed
model supports. Thinking-capable non-adaptive models (e.g. Haiku 4.5, Sonnet 4.5)
get the effort translated to a legacy thinking budget_tokens. Models with no
reasoning support have thinking/effort dropped under drop_params. And because
adaptive thinking carries no budget while the legacy form must satisfy Anthropic's
max_tokens > budget_tokens rule, the translated budget is capped below max_tokens,
dropping thinking when max_tokens can't fit the minimum budget. 4.6+ models pass
through untouched.

This matters because clients like Claude Code speak native Anthropic /v1/messages
and send the adaptive interface unconditionally, regardless of the routed model.
The native passthrough previously only capability-gated the OpenAI-style
reasoning_effort alias and forwarded native output_config/adaptive thinking raw, so
a pre-4.6 model rejected it with "This model does not support the effort parameter"
and the request failed. Claude Code already gets drop_params auto-set, so its
requests now succeed.

* test(anthropic): gate undersized-max_tokens thinking drop on drop_params; add edge tests

Addresses review feedback on the max_tokens-too-small branch. Previously a
thinking-capable model whose max_tokens could not fit the minimum thinking budget
had thinking silently dropped regardless of drop_params, while a residual
output_config field in the same call still raised when drop_params was off. Gate
both consistently on drop_params: raise a clear error (naming max_tokens for the
undersized case) when drop_params is off, drop otherwise. Claude Code gets
drop_params auto-set, so it still succeeds.

Adds tests for the undersized-max_tokens raise, the residual output_config raise,
and the no-adaptive-interface passthrough on a non-adaptive model.

* fix(anthropic): make adaptive-effort translation silent to avoid breaking provider strip contracts

The previous raise-when-not-drop_params behavior broke existing bedrock and vertex
messages tests: those providers already silently strip unsupported output_config
for pre-4.6 models (issue #22797) with no drop_params required, and the shared
parent transform raising pre-empted that. It also conflicted with the goal of
keeping requests working rather than failing them.

Make the reshape silent: translate effort to legacy thinking for thinking-capable
models, drop thinking for non-reasoning models, and remove only the consumed effort
key from output_config, leaving any residual (e.g. format) for provider subclasses
(bedrock/vertex) to handle. No raise, no drop_params gating. This also resolves the
review note about inconsistent drop_params handling by making every path uniform.

Updates the tests to assert the silent behavior and residual output_config
preservation.

* fix(anthropic): handle output_config-capable but non-adaptive models (Opus 4.5)

Greptile caught a real bug: the early-return guard treated supports_output_config
as equivalent to supporting adaptive thinking. Claude Opus 4.5 advertises
supports_output_config (it accepts output_config.effort) but is not adaptive, so it
rejects thinking:{type:adaptive} with "adaptive thinking is not supported on this
model". The guard early-returned for Opus 4.5 and forwarded the adaptive thinking
block raw, reproducing the exact failure the fix is meant to prevent.

thinking:{type:adaptive} and output_config.effort are independent capabilities.
Only early-return for adaptive-thinking models. For a model that supports
output_config.effort but is not adaptive, keep the native effort and drop only the
unsupported adaptive thinking block. Verified live against Opus 4.5: the Claude Code
payload now returns 200 instead of 400.

Adds regression tests for Opus 4.5 with and without adaptive thinking.

* fix(anthropic): translate adaptive thinking for effort-capable pre-4.6 models

Claude Opus 4.5 advertises supports_output_config but not adaptive thinking,
so the early-return guard forwarded thinking.type=adaptive raw and Anthropic
rejected it. The guard now only skips true adaptive models; effort-only
requests on effort-capable models still pass through untouched. The
_map_reasoning_effort call is wrapped to surface unrecognized effort values
as a clean 400, matching _translate_reasoning_effort_to_anthropic

* fix(anthropic): fall back to legacy thinking when effort level unsupported

Opus 4.5 accepts output_config.effort but only low/medium/high; Claude Code
defaults to xhigh on newer models, so preserving that level raw gets rejected
by Anthropic. Gate the native-effort passthrough on _validate_effort_for_model
and fall through to the budget translation for unsupported levels

* fix(anthropic): keep effort-only requests untouched for provider normalization

The xhigh fall-through consumed effort-only requests on effort-capable
models, breaking bedrock invoke's own normalization which clamps xhigh to
the model's ceiling after the base transform runs
(test_bedrock_messages_normalizes_output_config_effort_for_opus). Restrict
the fall-through to requests that carry adaptive thinking; effort-only
requests pass through so provider subclasses keep owning level clamping

---------

Co-authored-by: Abhimanyu Kapur <38531241+akapur99@users.noreply.github.com>
(cherry picked from commit 3a62e54)

* fix(bedrock): flag mapped Claude 4.8+ entries with supports_mid_conversation_system (#32882)

Exact cost-map hits resolve before fallback-generalization rules, so the
mapped Sonnet 5, Fable 5 and jp Opus 4.8 Bedrock entries bypassed the
bedrock-anthropic-claude-mid-conversation-system rule and hoisted
mid-conversation system messages, invalidating the prompt cache.

(cherry picked from commit c15891f)

* Merge pull request #32873 from BerriAI/litellm_fallback_rules_routing_split

refactor(fallback-generalizations): split rules into routing and provider-neutral capability kinds

(cherry picked from commit 45d3644)

* Merge pull request #32874 from BerriAI/litellm_thread_provider_capability_probes

fix(anthropic): thread real provider through capability probes instead of pinning anthropic

(cherry picked from commit ead7ad3)

* test: add /v1/messages to supported_endpoints schema enum (#32739)

(cherry picked from commit bf02a4a)

---------

Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Co-authored-by: Abhimanyu Kapur <38531241+akapur99@users.noreply.github.com>
Co-authored-by: tin-berri <tin@berri.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants