test(azure): guard the gpt-5 reasoning_effort=none base_model gate at the SDK layer

[mateo-berri]

Relevant issues

Guards #31243 at the SDK layer. The fix itself shipped in PR #28490; live e2e coverage of the same shape is in #33468

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

The guarded behavior is client-side SDK param mapping, so the before/after proof is the same real Azure OpenAI call made through litellm.completion on the last release without the fix versus this branch. The deployment gpt-5.6-sol-e2e is a real custom-named gpt-5.6-sol deployment on the e2e suite Azure resource; the after run reaches Azure and costs real $

Before, captured on litellm==1.86.7 from PyPI (the fix first shipped in v1.87.0; it entered release branches as a cherry-pick, so git tag --contains on the staging commit misleadingly points at v1.91.0):

$ python - <<'EOF'
import os
import litellm
litellm.completion(
    model="azure/gpt-5.6-sol-e2e",
    base_model="azure/gpt-5.6-sol",
    messages=[{"role": "user", "content": "A farmer has 17 sheep, all but 9 run away, then he buys twice as many as remain minus 3. How many sheep? Reply with just the number."}],
    max_completion_tokens=2000,
    reasoning_effort="none",
    api_base=os.environ["AZURE_API_BASE"],
    api_key=os.environ["AZURE_API_KEY"],
)
EOF
UnsupportedParamsError: litellm.UnsupportedParamsError: Azure OpenAI does not support reasoning_effort='none' for this model. Supported values are: 'low', 'medium', and 'high'. To drop this parameter, set `litellm.drop_params=True` or for proxy:

`litellm_settings:
 drop_params: true`
Issue: https://github.com/BerriAI/litellm/issues/16704

The error is raised before any network call, which is exactly why the e2e row in #33468 cannot see it: the proxy's config-file registration resolves capabilities via base_model on its own, so a live proxy passes this shape even on images whose SDK path is broken (verified on the v1.86.2 image)

After, captured at commit 2df2c6c (same call, printing the response):

$ python - <<'EOF'
import os
import litellm
r = litellm.completion(
    model="azure/gpt-5.6-sol-e2e",
    base_model="azure/gpt-5.6-sol",
    messages=[{"role": "user", "content": "A farmer has 17 sheep, all but 9 run away, then he buys twice as many as remain minus 3. How many sheep? Reply with just the number."}],
    max_completion_tokens=2000,
    reasoning_effort="none",
    api_base=os.environ["AZURE_API_BASE"],
    api_key=os.environ["AZURE_API_KEY"],
)
print(f"content={r.choices[0].message.content!r} reasoning_tokens={r.usage.completion_tokens_details.reasoning_tokens}")
EOF
content='24' reasoning_tokens=0

Type

✅ Test

Changes

Adds four tests to tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py (the test file PR #28490 introduced) pinning the exact GH #31243 shape at the SDK layer: reasoning_effort='none' against a custom-named Azure deployment whose true model is supplied via base_model

Two tests pin get_optional_params: with base_model="azure/gpt-5.6-sol" the param survives mapping as reasoning_effort='none', and without base_model the gate fails closed with UnsupportedParamsError, so the positive test cannot pass vacuously. Two tests pin the litellm.completion kwarg plumbing in main.py using mock_response, which short-circuits after get_optional_params runs, so they exercise the real param-mapping path with no network

Why this is needed on top of the e2e row in #33468: the proxy's startup registration of config-file models resolves capabilities through base_model independently, which masks an SDK-layer regression from any proxy-level test (the v1.86.2 image passes the e2e shape while its own SDK raises). These unit tests are the only guard that fails on an SDK-layer revert

Kill power was verified by mutation: reverting _azure_detection_model = base_model or model to model in litellm/utils.py fails 6 tests in the file including the new ones, and nulling the base_model kwarg extraction in litellm/main.py fails only the new completion-level test, coverage no existing test provided

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR adds four mock-only unit tests to test_azure_base_model_routing.py to pin the GH #31243 fix at the SDK layer — specifically that reasoning_effort='none' with a custom Azure deployment name is accepted when base_model points to a registered gpt-5 model, and rejected when base_model is absent.

• Two tests extend TestGetOptionalParamsWithBaseModel, covering the get_optional_params layer: one positive (resolves capability via base_model=\"azure/gpt-5.6-sol\") and one negative (raises UnsupportedParamsError without base_model).
• Two tests form a new TestCompletionThreadsBaseModelIntoParamGate class and exercise the same gate through litellm.completion using mock_response, verifying that base_model is correctly threaded through to param mapping without any real network calls.

Confidence Score: 4/5

Safe to merge; pure test addition with no production code changes and all tests use mocks.

Only change is adding four mock-based tests — no production code is touched. The tests are well-structured and the mock_response approach correctly short-circuits before any network call. The single observation is a mildly imprecise docstring on the negative test: gpt-5.6-sol-e2e actually does match gpt-5 detection (the 'gpt-5' in model substring check), so the gate fails at the capability-lookup step rather than at the detection step as the comment implies. The test still validates the correct contract, but a future contributor reading it cold could be misled about where exactly the guard fires.

The negative test docstring in test_azure_base_model_routing.py (lines 185-194) is slightly imprecise about the mechanism that triggers the rejection.

Important Files Changed

Filename	Overview
tests/test_litellm/llms/azure/chat/test_azure_base_model_routing.py	Adds four mock-only tests pinning the GH #31243 fix: two in TestGetOptionalParamsWithBaseModel for the get_optional_params layer, and two in a new TestCompletionThreadsBaseModelIntoParamGate class for the litellm.completion layer. Docstring on the negative test is slightly imprecise about why the gate fails closed.

_{Reviews (1): Last reviewed commit: "test(azure): guard the gpt-5 reasoning_e..." | Re-trigger Greptile}

[greptile-apps]

Misleading negative-test docstring

The docstring says "the 'none' gate fails closed for unknown deployment names", implying the model name bypasses gpt-5 detection. It doesn't — gpt-5.6-sol-e2e contains the literal "gpt-5" substring, so is_model_gpt_5_model returns True even without base_model. The UnsupportedParamsError is raised one step later: _supports_reasoning_effort_level("gpt-5.6-sol-e2e", "none") returns False because the custom deployment name is not registered in model_prices_and_context_window.json.

The test still validates the right contract (the gate must reject an unregistered deployment), but a reader might incorrectly conclude the guard operates at the detection step. A note like "name matches gpt-5 detection but the deployment is not registered to support none" would make the intent clearer for future contributors.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_unit_gpt5_base_model_gate (2df2c6c) with litellm_internal_staging (0d7b0f7)}

[devin-ai-integration]

🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it?

I read the description against our contribution rubric. Here's how it lined up:

What you got right:

• ✅ Linked a related GitHub issue
• ✅ Clear problem description
• ✅ Expected vs. actual behavior

What's still missing:

• end-to-end QA proof with a screenshot, video, or real commands plus output demonstrating the fix
• non-mocked proof that the change works against the real system

The PR has solid context and a clear before/after problem description, but the only evidence is pasted command output from a real call plus discussion of unit tests; the body does not include a screenshot/video, and the commands shown are not clearly presented as a reproducible end-to-end QA artifact for the PR itself. Since QA proof is required and unit tests do not count, this fails triage.

If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close.

During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof).

If the PR does get auto-closed in 24 hours, you still have easy recovery paths:

• Comment @agent-shin reconsider after updating the description. I'll re-evaluate and reopen the PR if it now passes.
• Comment @greptileai to request a fresh Greptile review; that still works even after the PR is closed, and a stronger score is one of the signals that lifts the PR back into the queue. So a low Greptile score isn't a blocker either.

Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer.

(I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.)