Skip to content

test: assert the prefixed model the azure responses bridge now hands back - #37749

Merged
yuneng-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_azure_responses_bridge_assertion
Aug 21, 2026
Merged

test: assert the prefixed model the azure responses bridge now hands back#37749
yuneng-berri merged 1 commit into
litellm_internal_stagingfrom
litellm_azure_responses_bridge_assertion

Conversation

@yuneng-berri

@yuneng-berri yuneng-berri commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • Expect azure/test-azure-computer-use-preview, matching the bridge
  • One expected value moves, nothing else

User Flow

This PR touches a test file only, so no end user flow changes. What changes is the model id one assertion expects the chat-to-responses bridge to hand back

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

This is a test-only assertion change and the suite it lives in needs azure credentials that only CI holds, so CI is where both sides have to be captured

Before (6eacdbf)

ci/circleci: llm_translation_testing

  1. test_azure_openai_responses_bridge fails at tests/llm_translation/test_azure_openai.py:651
  2. AssertionError: assert 'azure/test-azure-computer-use-preview' == 'test-azure-computer-use-preview'
  3. Captured on litellm_internal_staging at cc812cdf, an ancestor of this merge base that carries the identical assertion: https://app.circleci.com/workflow/e36bb1a2-ff9a-4587-ba96-7817672f1780
  4. It is the only test_azure_openai failure in that job, so nothing else in the file is implicated

After (461aa19)

ci/circleci: llm_translation_testing

Run triggered by the run-ci label on this commit: https://app.circleci.com/workflow/5f3ebfc8-bc03-4930-a2f2-3f04e4dc7b46

  1. test_azure_openai_responses_bridge passes, so the assertion now matches what the bridge hands back
  2. The job is still red, on 16 TestTogetherAI failures that have nothing to do with this change
  3. Those are the same live together_ai calls test: point the live together_ai suites at a model together still serves #37746 repoints, and they fail here because this branch sits on the base that still names the retired model
  4. So the two PRs are each red on what the other fixes, and this job goes green once test: point the live together_ai suites at a model together still serves #37746 lands and staging merges back in

Type

✅ Test

Caveats (if any)

…back

a369cb0 made completion() hand the prefixed model back to responses(), so
that responses() running get_llm_provider() a second time becomes a no-op
instead of stripping a prefix the model id owns. That was deliberate, and it
shipped with its own unit test, but it left two older assertions behind still
expecting the bare id.

#37744 corrected the openai one in test_openai.py. This is its azure sibling,
which llm_translation_testing has been failing on ever since.

Only the expected value moves. The neighbouring custom_llm_provider assertion
already passes and stays as it is.
@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Updates one Azure responses-bridge test assertion to match the bridge’s intentional provider-prefixed model output.

  • Expects azure/test-azure-computer-use-preview instead of the bare deployment name.
  • Leaves the separate Azure provider assertion unchanged.

Confidence Score: 5/5

The PR appears safe to merge because the test-only change aligns the assertion with the bridge’s intentional model-prefix behavior.

The bridge deliberately restores the Azure provider prefix before invoking the responses path, so the updated expectation accurately tests existing production behavior and introduces no blocking failure.

Important Files Changed

Filename Overview
tests/llm_translation/test_azure_openai.py The assertion now matches the production bridge contract without weakening coverage or changing runtime behavior.

Reviews (1): Last reviewed commit: "test: assert the prefixed model the azur..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@yuneng-berri
yuneng-berri enabled auto-merge (squash) August 21, 2026 01:51
@yuneng-berri
yuneng-berri merged commit 60225ab into litellm_internal_staging Aug 21, 2026
111 of 115 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_azure_responses_bridge_assertion branch August 21, 2026 01:53

@tin-berri tin-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct fix — traced this to _restore_routing_prefix() in litellm/completion_extras/litellm_responses_transformation/handler.py, which already prefixes the model with its provider (f"{custom_llm_provider}/{model}") before handing off to litellm.responses(), specifically so responses()'s own get_llm_provider() call gets back the prefix completion() started from. The updated assertion matches that already-shipped production behavior; Greptile's independent review reached the same conclusion. The 3 CircleCI failures (llm_translation_testing, local_testing_part1/2) don't look caused by this 1-line test-only change — local_testing doesn't even touch the modified file — and I couldn't find any indication in the diff or surrounding code that would make the new assertion wrong. Approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants