test: assert the prefixed model the azure responses bridge now hands back - #37749
Conversation
…back a369cb0 made completion() hand the prefixed model back to responses(), so that responses() running get_llm_provider() a second time becomes a no-op instead of stripping a prefix the model id owns. That was deliberate, and it shipped with its own unit test, but it left two older assertions behind still expecting the bare id. #37744 corrected the openai one in test_openai.py. This is its azure sibling, which llm_translation_testing has been failing on ever since. Only the expected value moves. The neighbouring custom_llm_provider assertion already passes and stays as it is.
Greptile SummaryUpdates one Azure responses-bridge test assertion to match the bridge’s intentional provider-prefixed model output.
Confidence Score: 5/5The PR appears safe to merge because the test-only change aligns the assertion with the bridge’s intentional model-prefix behavior. The bridge deliberately restores the Azure provider prefix before invoking the responses path, so the updated expectation accurately tests existing production behavior and introduces no blocking failure.
|
| Filename | Overview |
|---|---|
| tests/llm_translation/test_azure_openai.py | The assertion now matches the production bridge contract without weakening coverage or changing runtime behavior. |
Reviews (1): Last reviewed commit: "test: assert the prefixed model the azur..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
60225ab
into
litellm_internal_staging
tin-berri
left a comment
There was a problem hiding this comment.
Correct fix — traced this to _restore_routing_prefix() in litellm/completion_extras/litellm_responses_transformation/handler.py, which already prefixes the model with its provider (f"{custom_llm_provider}/{model}") before handing off to litellm.responses(), specifically so responses()'s own get_llm_provider() call gets back the prefix completion() started from. The updated assertion matches that already-shipped production behavior; Greptile's independent review reached the same conclusion. The 3 CircleCI failures (llm_translation_testing, local_testing_part1/2) don't look caused by this 1-line test-only change — local_testing doesn't even touch the modified file — and I couldn't find any indication in the diff or surrounding code that would make the new assertion wrong. Approved.
TLDR
Problem this solves:
llm_translation_testingis red on one azure testHow it solves it:
azure/test-azure-computer-use-preview, matching the bridgeUser Flow
This PR touches a test file only, so no end user flow changes. What changes is the model id one assertion expects the chat-to-responses bridge to hand back
Relevant issues
Linear ticket
Pre-Submission checklist
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
This is a test-only assertion change and the suite it lives in needs azure credentials that only CI holds, so CI is where both sides have to be captured
Before (6eacdbf)
ci/circleci: llm_translation_testing
test_azure_openai_responses_bridgefails attests/llm_translation/test_azure_openai.py:651AssertionError: assert 'azure/test-azure-computer-use-preview' == 'test-azure-computer-use-preview'litellm_internal_stagingatcc812cdf, an ancestor of this merge base that carries the identical assertion: https://app.circleci.com/workflow/e36bb1a2-ff9a-4587-ba96-7817672f1780test_azure_openaifailure in that job, so nothing else in the file is implicatedAfter (461aa19)
ci/circleci: llm_translation_testing
Run triggered by the
run-cilabel on this commit: https://app.circleci.com/workflow/5f3ebfc8-bc03-4930-a2f2-3f04e4dc7b46test_azure_openai_responses_bridgepasses, so the assertion now matches what the bridge hands backTestTogetherAIfailures that have nothing to do with this changeType
✅ Test
Caveats (if any)
custom_llm_providerassertion on the next line stays untouched