fix(azure_ai): advertise 1M context window for Claude Opus 4.6+ on Foundry

[devin-ai-integration]

TLDR

Problem this solves:

• Azure Claude Opus 4.6/4.7/4.8 advertised a 200k context
• Clients like Claude Code CLI compacted prompts unnecessarily

How it solves it:

• Bump max_input_tokens to 1000000 for those azure_ai entries
• Pin the values in a cost-map regression test

Relevant issues

Linear ticket

Resolves LIT-4406

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Anthropic's docs state that Opus 4.6, Opus 4.7, Opus 4.8, Sonnet 5 and Sonnet 4.6 all serve a 1M-token context window on the Claude API, Amazon Bedrock, Google Cloud and Microsoft Foundry, with 1M as the default and no beta header needed, so the azure_ai entries were the outliers; Sonnet 5 and Sonnet 4.6 were already correct on our side

Proxy config used for both runs (three Foundry Opus deployments; /model/info is metadata-only, so no provider call is made):

model_list:
  - model_name: claude-opus-4-8
    litellm_params:
      model: azure_ai/claude-opus-4-8
      api_key: fake
      api_base: https://example.openai.azure.com
  - model_name: claude-opus-4-7
    litellm_params: {model: azure_ai/claude-opus-4-7, api_key: fake, api_base: https://example.openai.azure.com}
  - model_name: claude-opus-4-6
    litellm_params: {model: azure_ai/claude-opus-4-6, api_key: fake, api_base: https://example.openai.azure.com}

Before, at 593b12dc56~1:

$ LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config azcfg.yaml --port 4000
$ curl -s http://localhost:4000/v1/model/info -H "Authorization: Bearer sk-1234" \
    | jq -r '.data[] | "\(.model_name) \(.model_info.max_input_tokens)"'
claude-opus-4-8 200000
claude-opus-4-7 200000
claude-opus-4-6 200000

After, at 593b12dc56:

$ curl -s http://localhost:4000/v1/model/info -H "Authorization: Bearer sk-1234" \
    | jq -r '.data[] | "\(.model_name) \(.model_info.max_input_tokens) \(.model_info.max_output_tokens)"'
claude-opus-4-8 1000000 128000
claude-opus-4-7 1000000 128000
claude-opus-4-6 1000000 128000

Live proof that Foundry actually serves >200k, captured at head e6b5511dcf

The runs above only prove the metadata change, so this run pays real money to answer the question that matters: do live Foundry deployments accept prompts past the old 200k advertisement with no anthropic-beta header? Proxy at this PR's head, config pointing at two real Foundry resources with no extra_headers anywhere:

model_list:
  - model_name: claude-opus-4-8
    litellm_params:
      model: azure_ai/claude-opus-4-8
      api_base: os.environ/AZURE_FOUNDRY_API_BASE
      api_key: os.environ/AZURE_FOUNDRY_API_KEY
  - model_name: claude-opus-4-7
    litellm_params:
      model: azure_ai/claude-opus-4-7
      api_base: os.environ/AZURE_AI_API_BASE
      api_key: os.environ/AZURE_AI_API_KEY
  - model_name: claude-opus-4-6
    litellm_params:
      model: azure_ai/claude-opus-4-6
      api_base: os.environ/AZURE_FOUNDRY_API_BASE
      api_key: os.environ/AZURE_FOUNDRY_API_KEY

Each request puts a needle in the first line, pads the prompt well past 200k tokens with filler, and asks for the needle back:

$ LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config azcfg.yaml --port 36859
$ block='The quarterly logistics report covers inventory turnover, freight consolidation, warehouse utilization, and carrier performance metrics across all regional distribution centers. Procurement noted that supplier lead times improved by four days after the new vendor scorecard rollout in the northeast corridor. Fleet maintenance costs held steady while fuel hedging contracts offset most of the seasonal price variance observed last quarter. Customer order accuracy reached record levels following the barcode validation upgrade deployed across the packing lines. The demand planning team recalibrated safety stock thresholds for slow-moving SKUs to reduce carrying costs without hurting fill rates. '
$ for m in claude-opus-4-6 claude-opus-4-7 claude-opus-4-8; do
    printf "${block}%.0s" {1..1920} | jq -Rs --arg m "$m" '{model:$m, messages:[{role:"user", content:("The project codename is quixotic-heron-742.\n\n" + . + "\n\nWhat is the project codename mentioned at the beginning of this message? Reply with only the codename.")}], max_tokens:30}' > req.json
    echo "== $m"
    curl -s -D h.txt http://localhost:36859/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" --data-binary @req.json -o resp.json
    grep -i '^x-litellm-response-cost:' h.txt
    jq -c '{answer:.choices[0].message.content, finish:.choices[0].finish_reason, prompt_tokens:.usage.prompt_tokens, completion_tokens:.usage.completion_tokens}' resp.json
  done
== claude-opus-4-6
x-litellm-response-cost: 1.1525
{"answer":"quixotic-heron-742","finish":"stop","prompt_tokens":230445,"completion_tokens":11}
== claude-opus-4-7
x-litellm-response-cost: 2.0646100000000005
{"answer":"quixotic-heron-742","finish":"stop","prompt_tokens":412857,"completion_tokens":13}
== claude-opus-4-8
x-litellm-response-cost: 2.064535
{"answer":"quixotic-heron-742","finish":"stop","prompt_tokens":412852,"completion_tokens":11}

All three deployments, spanning two independent Foundry resources, accept 230k to 413k input tokens with no beta header, retrieve the needle placed at position zero, and finish with stop. The tracked cost matches the flat rate exactly (for Opus 4.6, 230445 input tokens at $5/M plus 11 output tokens at $25/M comes to $1.1525), so there is no hidden long-context pricing tier to model. One earlier attempt that padded with a single sentence repeated thousands of times tripped the provider content filter on claude-opus-4-8 (finish_reason: content_filter); even that request was accepted and metered at 499,254 input tokens, so no context-window rejection occurred at any size sent

Type

🐛 Bug Fix

Changes

azure_ai/claude-opus-4-6, azure_ai/claude-opus-4-7 and azure_ai/claude-opus-4-8 move from max_input_tokens: 200000 to 1000000 in both the root cost map and the bundled backup. Two existing tests deliberately pinned the 200k value (one with a comment claiming Foundry caps Opus 4.8 at 200k), so those assertions are corrected, and tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py gains a test that pins the full Foundry Claude split: 1M for Opus 4.6+/Opus 5/Sonnet 5/Sonnet 4.6, still 200k for Opus 4.1, Opus 4.5, Sonnet 4.5 and Haiku 4.5

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/8ff9f37120754d5299837b9815ed3d5e
Requested by: @shivamrawat1

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

Updates Azure AI metadata to advertise a 1M-token input context for Claude Opus 4.6, 4.7, and 4.8.

• Synchronizes the root and bundled backup cost maps.
• Updates existing Opus configuration expectations.
• Parametrizes the regression matrix over both maps, fully addressing the previous review thread.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains, and the previous root-map coverage gap is fully addressed by independently testing both shipped maps.

Important Files Changed

Filename	Overview
model_prices_and_context_window.json	Raises the three Azure AI Opus context-window values from 200k to 1M in the root runtime map.
litellm/model_prices_and_context_window_backup.json	Mirrors the same context-window updates in the independently loaded bundled fallback map.
tests/test_litellm/litellm_core_utils/test_get_model_cost_map.py	Checks the complete Foundry Claude context-window matrix independently against both the root and bundled maps.
tests/test_litellm/test_claude_opus_4_6_config.py	Updates the Opus 4.6 Azure AI capability expectation to the corrected 1M-token limit.
tests/test_litellm/test_claude_opus_4_8_config.py	Updates the Opus 4.8 Azure AI capability expectation and removes the obsolete 200k-cap claim.

_{Reviews (2): Last reviewed commit: "test(cost_map): cover root map in the Fo..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[shivamrawat1]

@greptile review again

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_azure_claude_1m_context (e6b5511) with litellm_internal_staging (8177230)}

[mateo-berri]

Tested a >200k query and it works. LGTM