fix(fireworks_ai): correct glm-5p2 context window to 1,040,000 tokens - #30886
mateo-berri wants to merge 1 commit into
Conversation
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@mateo-berri the Fireworks AI api supports 1048575 context window size for the GLM-5.2 model. The model card lists it as 1048576 on https://fireworks.ai/models
#31460 corrects the context window for the model based on what the API supports and also adds its Fast variant. It also has a sample curl request and its screenshot with the API error providing the 1046757 max context length error. The GLM 5.2 base model's cache read price has also been lowered. 1040K context window size seems to be the model page listing it incorrectly, will report that to the team. |
|
🚅 Hi, thanks for the PR! I'm Agent Shin, the automated triage bot for this repository. What's this and why am I getting it? I read the description against our contribution rubric. Here's how it lined up: What you got right:
What's still missing:
If the description isn't updated in the next 24 hours, I'll auto-close this PR. That's not us saying we don't care about the change; we want the open-PR list to mirror what a maintainer can act on right now, so contributors don't get lost in a backlog. A closed PR is a soft "park this for later," not a rejection. Take your time; everything below still works after the close. During the grace period: just update the PR description with the missing pieces. No need to ping me; I'll re-check on the next sweep and skip the auto-close if it now passes. See what counts as QA proof for the full rubric (a linked issue alone isn't enough; it covers context, not proof). If the PR does get auto-closed in 24 hours, you still have easy recovery paths:
Internal BerriAI contributors: this rubric doesn't apply to you; ping a maintainer. (I'm an LLM, so I'm not infallible. If you think I got this wrong, ping a maintainer; they'll override me.) |
|
Superseded by rolling registry audit #37902, which reverified and absorbed the provider-documented metadata changes at field level |

Relevant issues
Slack request to get Fireworks GLM 5.2 into internal staging. The model was already present in the cost map (added during the OSS sync in #30637) with correct per-token pricing, but its context window was off, so this cleans that up.
Linear ticket
n/a
Pre-Submission checklist
make test-unitChanges
fireworks_ai/.../glm-5p2was mapped withmax_input_tokens: 1048576(2^20). Fireworks publishes the GLM 5.2 context length as "1040k tokens", which is 1,040,000, and their listings truncate to the nearest thousand (GLM 5.1 shows "202k" for the mapped 202800). A value of 1,048,576 would have surfaced as "1048k", so it does not match the real limit and would let LiteLLM accept requests between 1,040,000 and 1,048,576 tokens that Fireworks rejects.This corrects
max_input_tokensto 1,040,000 for both the long-form (fireworks_ai/accounts/fireworks/models/glm-5p2) and short-form (fireworks_ai/glm-5p2) keys inmodel_prices_and_context_window.jsonand the bundledlitellm/model_prices_and_context_window_backup.json, and updates the matching expectation in the Fireworks cost-map test so the value is now guarded against regressing. Pricing ($1.40 / $0.26 / $4.40 per 1M input/cached/output), function calling, reasoning andsupports_vision: falsealready matched the Fireworks docs and are unchanged. No-fastrouter variant is added because Fireworks lists Fast and Priority tiers as "coming soon" for GLM 5.2, so that path does not exist yet.Source: https://fireworks.ai/models/fireworks/glm-5p2 and https://docs.fireworks.ai/serverless/pricing
Screenshots / Proof of Fix
Ran the proxy against the edited local cost map (
LITELLM_LOCAL_MODEL_COST_MAP=True) with a one-model config pointing atfireworks_ai/accounts/fireworks/models/glm-5p2, then queried/v1/model/info:Regression test (would fail if the JSON drifts from 1,040,000):
A real billed completion was not run here because no
FIREWORKS_AI_API_KEYis configured in this environment. To verify end to end against the live Fireworks API, run the proxy with a key set and curl a chat completion:Type
🐛 Bug Fix
Slack Thread