Skip to content

fix(catalog): honour a member's max_output_tokens override in combo metadata - #16556

Merged
diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.52from
pacocartones:fix/16515-combo-max-output-override
Oct 15, 2026
Merged

diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.52from
pacocartones:fix/16515-combo-max-output-override

Conversation

@pacocartones

Copy link
Copy Markdown
Contributor

Summary

  • getComboTargetCatalogMetadata (src/app/api/v1/models/catalog.ts) took a combo member's max_output_tokens from the raw synced row (synced.limit_output). An operator's max_output_tokens override in model_capability_overrides reached the per-model catalog entry but not the combo built from it, so the combo advertised the pre-override minimum.
  • canonical.limits.maxOutputTokens, resolved a few lines above, already applies the override. max_input_tokens already reads canonical.limits. Output now does the same, with the synced row and the static spec kept as fallbacks.
  • With the issue's shape (override 65536 on a member whose synced row says 1048576, plus a member at 384000), the combo goes from max_output_tokens: 384000 to 65536.

Related Issues

Target Branch

  • Base is the active release/v3.8.x branch (the highest open release/v*), not main
  • New feature? From 3.8.55 on it is held with the v4-feature label for the v4 channel

Validation

  • Change type: other (models catalog)
  • Focused tests and category gates from the golden path
  • npm run lint (eslint on the changed files: no new findings; the existing no-unused-vars findings in catalog.ts are on untouched lines and also show on the base)
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/16515-combo-max-output-override.test.ts (new): fails on the base (actual: 384000, expected: 65536) and passes with the fix.

Coverage Notes

  • With the fix, the new test plus models-catalog-route, auto-combo-context-advertising, models-catalog-hidden-combo-leaves and combo-context-prefix-resolution give 70/70 passing.

Reviewer Notes

  • When no override is set, canonical.limits.maxOutputTokens takes the synced row first, as before. It also takes the registry model's value before the static spec, which is the order the per-model entry already uses.

…etadata

getComboTargetCatalogMetadata read max_output_tokens from the raw synced row,
so an operator's max_output_tokens override reached the per-model catalog
entry but not the combo built from it. Read canonical.limits.maxOutputTokens
first, as max_input_tokens already does, and keep the synced row and the
static spec as fallbacks.

Fixes cheaper-inference#16515
@diegosouzapw
diegosouzapw merged commit 3d458de into cheaper-inference:release/v3.8.52 Oct 15, 2026
31 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(api): Combo catalog metadata ignores max_output_tokens capability overrides

2 participants