Skip to content

fix(fireworks_ai): correct glm-5p2 context window to 1,040,000 tokens - #30886

Closed
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fireworks-glm-5p2-context-window-1799
Closed

mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fireworks-glm-5p2-context-window-1799

Conversation

@mateo-berri

Copy link
Copy Markdown
Contributor

Relevant issues

Slack request to get Fireworks GLM 5.2 into internal staging. The model was already present in the cost map (added during the OSS sync in #30637) with correct per-token pricing, but its context window was off, so this cleans that up.

Linear ticket

n/a

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem

Changes

fireworks_ai/.../glm-5p2 was mapped with max_input_tokens: 1048576 (2^20). Fireworks publishes the GLM 5.2 context length as "1040k tokens", which is 1,040,000, and their listings truncate to the nearest thousand (GLM 5.1 shows "202k" for the mapped 202800). A value of 1,048,576 would have surfaced as "1048k", so it does not match the real limit and would let LiteLLM accept requests between 1,040,000 and 1,048,576 tokens that Fireworks rejects.

This corrects max_input_tokens to 1,040,000 for both the long-form (fireworks_ai/accounts/fireworks/models/glm-5p2) and short-form (fireworks_ai/glm-5p2) keys in model_prices_and_context_window.json and the bundled litellm/model_prices_and_context_window_backup.json, and updates the matching expectation in the Fireworks cost-map test so the value is now guarded against regressing. Pricing ($1.40 / $0.26 / $4.40 per 1M input/cached/output), function calling, reasoning and supports_vision: false already matched the Fireworks docs and are unchanged. No -fast router variant is added because Fireworks lists Fast and Priority tiers as "coming soon" for GLM 5.2, so that path does not exist yet.

Source: https://fireworks.ai/models/fireworks/glm-5p2 and https://docs.fireworks.ai/serverless/pricing

Screenshots / Proof of Fix

Ran the proxy against the edited local cost map (LITELLM_LOCAL_MODEL_COST_MAP=True) with a one-model config pointing at fireworks_ai/accounts/fireworks/models/glm-5p2, then queried /v1/model/info:

$ curl -s http://localhost:4000/v1/model/info -H "Authorization: Bearer sk-1234"
model_name       : fireworks-glm-5p2
litellm model    : fireworks_ai/accounts/fireworks/models/glm-5p2
max_input_tokens : 1040000
max_output_tokens: 131072
input_cost/token : 1.4e-06
output_cost/token: 4.4e-06
cache_read_cost  : 2.6e-07
supports_fn_call : True
supports_vision  : False

Regression test (would fail if the JSON drifts from 1,040,000):

$ uv run pytest tests/test_litellm/test_utils.py -k fireworks -q
3 passed, 204 deselected
$ uv run pytest tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py -q
23 passed

A real billed completion was not run here because no FIREWORKS_AI_API_KEY is configured in this environment. To verify end to end against the live Fireworks API, run the proxy with a key set and curl a chat completion:

export FIREWORKS_AI_API_KEY=...   # real key
litellm --config config.yaml --port 4000   # config maps fireworks-glm-5p2 -> fireworks_ai/accounts/fireworks/models/glm-5p2

curl -s http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" \
  -H "Content-Type: application/json" \
  -d '{"model":"fireworks-glm-5p2","messages":[{"role":"user","content":"In one sentence, what is GLM 5.2?"}]}'

Type

🐛 Bug Fix

Slack Thread

Open in Web Open in Cursor 

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov

codecov Bot commented Jun 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@shzdehmd

Copy link
Copy Markdown
Contributor

@mateo-berri the Fireworks AI api supports 1048575 context window size for the GLM-5.2 model. The model card lists it as 1048576 on https://fireworks.ai/models

image

#31460 corrects the context window for the model based on what the API supports and also adds its Fast variant. It also has a sample curl request and its screenshot with the API error providing the 1046757 max context length error. The GLM 5.2 base model's cache read price has also been lowered.

1040K context window size seems to be the model page listing it incorrectly, will report that to the team.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it?

I read the description against our contribution rubric. Here's how it lined up:

What you got right:

  • ✅ Linked a related GitHub issue
  • ✅ Clear problem description
  • ✅ Expected vs. actual behavior

What's still missing:

  • end-to-end QA proof with real output from a non-mocked live system run
  • screenshot or video showing the fix working

The PR has solid context and a clear bug description with expected vs. actual behavior, but the only concrete verification shown is unit-test output and a hypothetical command for a live API call. Since unit tests are not end-to-end proof and no real live-system output, screenshot, or video is included, it fails triage.

If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close.

During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof).

If the PR does get auto-closed in 24 hours, you still have easy recovery paths:

  • Comment @agent-shin reconsider after updating the description. I'll re-evaluate and reopen the PR if it now passes.
  • Comment @greptileai to request a fresh Greptile review; that still works even after the PR is closed, and a stronger score is one of the signals that lifts the PR back into the queue. So a low Greptile score isn't a blocker either.

Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer.

(I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.)

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Superseded by rolling registry audit #37902, which reverified and absorbed the provider-documented metadata changes at field level

@devin-ai-integration
devin-ai-integration Bot deleted the litellm_fireworks-glm-5p2-context-window-1799 branch September 10, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants