Skip to content

feat(models): sync together_ai model registry - #38733

Closed
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_together_registry_sync_2026-08-29
Closed

feat(models): sync together_ai model registry#38733
mateo-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_together_registry_sync_2026-08-29

Conversation

@mateo-berri

Copy link
Copy Markdown
Contributor

Automated daily sync of the together_ai entries in model_prices_and_context_window.json against GET https://api.together.ai/v1/models?serverless and https://docs.together.ai/docs/deprecations.md by scripts/sync_together_ai_models.py.

Added (1)

  • together_ai/zai-org/GLM-5.3

Updated (1)

  • together_ai/Qwen/Qwen3.8-2.4T-A95B: input_cost_per_token: 2e-06 -> 2.5e-06; output_cost_per_token: 6e-06 -> 6.25e-06; cache_read_input_token_cost: 2.5e-07 -> 5e-07

Marked deprecated (0)

  • none

Returned to the catalog (0)

  • none

Warnings needing a human call (8)

  • together_ai/zai-org/GLM-5.3 added without a capability rule; review its tools/vision/reasoning support and add one
  • together_ai/BAAI/bge-base-en-v1.5 is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/Qwen/Qwen2.5-7B-Instruct-Turbo is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/baai/bge-base-en-v1.5 is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/deepseek-ai/DeepSeek-V3 is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/moonshotai/Kimi-K2-Instruct is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/togethercomputer/CodeLlama-34b-Instruct is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call
  • together_ai/zai-org/GLM-4.6 is absent from the serverless catalog with no removal date in the docs; needs a human deprecation call

Catalog model types outside the sync's token-pricing scope, skipped: audio (5), image (29), transcribe (4), video (38)

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR synchronizes Together AI model metadata in both copies of the model registry.

  • Adds the GLM-5.3 catalog entry with pricing, caching, and token-limit metadata.
  • Updates Qwen3.8 input, output, and cached-input pricing.
  • Keeps the primary and backup registries aligned.

Confidence Score: 4/5

The GLM-5.3 output limit should be corrected before merging because the current metadata permits requests that the upstream model rejects.

The new entry conflates GLM-5.3's one-million-token context window with its 128K output ceiling, causing local validation and token budgeting to accept unsupported output sizes.

Files Needing Attention: model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds GLM-5.3 and updates Qwen3.8 pricing, but overstates GLM-5.3's maximum output length by using its full context window.
litellm/model_prices_and_context_window_backup.json Mirrors the primary registry changes, including the incorrect GLM-5.3 maximum output-token value.

Reviews (1): Last reviewed commit: "feat(models): sync together_ai model reg..." | Re-trigger Greptile

"input_cost_per_token": 1.4e-06,
"litellm_provider": "together_ai",
"max_input_tokens": 1048575,
"max_output_tokens": 1048575,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 GLM-5.3 output limit overstated

When a GLM-5.3 request specifies more than 131072 output tokens, this metadata treats the model’s full context window as its output ceiling, causing LiteLLM to accept and forward a request that the upstream model rejects.

Knowledge Base Used: Provider adapters and capabilities

@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_together_registry_sync_2026-08-29 (994b2cf) with litellm_internal_staging (4d71441)

Open in CodSpeed

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Superseded by the rolling registry PR #38693, which absorbs this sync's GLM-5.3 addition (with a max_output_tokens correction to 128000 and capability flags) and the Qwen3.8 repricing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants