feat(models): sync together_ai model registry - #38733
Conversation
|
|
Greptile SummaryThe PR synchronizes Together AI model metadata in both copies of the model registry.
Confidence Score: 4/5The GLM-5.3 output limit should be corrected before merging because the current metadata permits requests that the upstream model rejects. The new entry conflates GLM-5.3's one-million-token context window with its 128K output ceiling, causing local validation and token budgeting to accept unsupported output sizes. Files Needing Attention: model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds GLM-5.3 and updates Qwen3.8 pricing, but overstates GLM-5.3's maximum output length by using its full context window. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the primary registry changes, including the incorrect GLM-5.3 maximum output-token value. |
Reviews (1): Last reviewed commit: "feat(models): sync together_ai model reg..." | Re-trigger Greptile
| "input_cost_per_token": 1.4e-06, | ||
| "litellm_provider": "together_ai", | ||
| "max_input_tokens": 1048575, | ||
| "max_output_tokens": 1048575, |
There was a problem hiding this comment.
GLM-5.3 output limit overstated
When a GLM-5.3 request specifies more than 131072 output tokens, this metadata treats the model’s full context window as its output ceiling, causing LiteLLM to accept and forward a request that the upstream model rejects.
Knowledge Base Used: Provider adapters and capabilities
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Superseded by the rolling registry PR #38693, which absorbs this sync's GLM-5.3 addition (with a max_output_tokens correction to 128000 and capability flags) and the Qwen3.8 repricing. |
Automated daily sync of the together_ai entries in model_prices_and_context_window.json against
GET https://api.together.ai/v1/models?serverlessand https://docs.together.ai/docs/deprecations.md by scripts/sync_together_ai_models.py.Added (1)
together_ai/zai-org/GLM-5.3Updated (1)
together_ai/Qwen/Qwen3.8-2.4T-A95B: input_cost_per_token: 2e-06 -> 2.5e-06; output_cost_per_token: 6e-06 -> 6.25e-06; cache_read_input_token_cost: 2.5e-07 -> 5e-07Marked deprecated (0)
Returned to the catalog (0)
Warnings needing a human call (8)
together_ai/zai-org/GLM-5.3added without a capability rule; review its tools/vision/reasoning support and add onetogether_ai/BAAI/bge-base-en-v1.5is absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/Qwen/Qwen2.5-7B-Instruct-Turbois absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/baai/bge-base-en-v1.5is absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/deepseek-ai/DeepSeek-V3is absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/moonshotai/Kimi-K2-Instructis absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/togethercomputer/CodeLlama-34b-Instructis absent from the serverless catalog with no removal date in the docs; needs a human deprecation calltogether_ai/zai-org/GLM-4.6is absent from the serverless catalog with no removal date in the docs; needs a human deprecation callCatalog model types outside the sync's token-pricing scope, skipped: audio (5), image (29), transcribe (4), video (38)