Repository navigation
fix(providers): pin uncloseai's context window at the provider level, not the model id - #17039
Closed
opensource-elearning wants to merge 1 commit into
Conversation
… not the model id
uncloseai rotates its /v1/models roster every few weeks. resolveTokenLimit()
looked the window up by model id, so a freshly rotated id missed the registry and
fell through to the generic 128000 catch-all; enforceOutputTokenBudget() then
derived available output from 128k and sent max_tokens 32000 against a 33537
token prompt, so the upstream vLLM rejected the request ("maximum context length
is 65536 tokens ... you requested 65537").
A provider-level defaultContextLength survives the rotation: any uncloseai model
id resolves to the real 65536 window with no discovery run, while per-model
discovery (max_model_len -> auto:discovery override) still wins once a sync has
run. This removes the hand-picked per-provider output cap the failure otherwise
tempts you into adding.
Refresh the pinned seed, free-model catalog row and authHint to the live id
turboderp/Qwen3.8-27B-exl3, and guard the new behaviour with tests that cover
both a retired id and a not-yet-seen one.
Rebased onto release/v3.8.52:
- tests/unit/models-catalog-route.test.ts and tests/unit/combo-builder-options-route.test.ts
asserted the retired id, display name and root; they now assert the live ones.
Those three assertions were red on this branch before the change.
- docs/reference/PROVIDER_REFERENCE.md regenerated from the updated authHint
(npm run gen:provider-reference) — only the uncloseai row and the generated
date lines move.
Verified:
- node --import tsx/esm --test tests/unit/models-catalog-route.test.ts tests/unit/combo-builder-options-route.test.ts tests/unit/uncloseai-registry-refresh.test.ts
-> 53/53 pass
- npx vitest run open-sse/services/__tests__/tierResolver.test.ts -> 37/37 pass
- GET https://hermes.ai.unturf.com/v1/models serves exactly one model:
id turboderp/Qwen3.8-27B-exl3, root turboderp/Qwen3.8-27B-exl3, max_model_len 65536.
The retired id is no longer served.
- npm run typecheck:core -> clean
- npx eslint on the changed files with --suppressions-location config/quality/eslint-suppressions.json
(the CI invocation) -> clean
Contributor
Author
|
Superseded by #17067, which re-applies this fix on the current This branch was cut before the base moved, and the base has since rewritten files this change touches (the no-auth allowlist test pair), so the PR no longer merges cleanly. #17067 carries the same fix — provider-level Closing this one to keep a single open PR for the change. |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #17038
Problem
UncloseAI serves exactly one model and rotates that roster every few weeks. The registry pinned a single id and the provider declared no provider-level context window, so after a rotation two things break:
/v1/chat/completions, andresolveTokenLimit(), which falls through to the generic128000catch-all —enforceOutputTokenBudget()then derives an output budget larger than the real 65536 window, and the upstream vLLM rejects the request (maximum context length is 65536 tokens … you requested 65537).Change
open-sse/config/providers/registry/uncloseai/index.ts—models[]points at the live id, and the window is pinned at the provider level (defaultContextLength: 65536) so it survives the next rotation. Per-model discovery (max_model_len→auto:discovery) still wins once a sync has run.open-sse/config/freeModelCatalog.data.ts— catalog row refreshed to the live id, with the refresh-day comment.src/shared/constants/providers/noauth.ts—authHintcites the live model.tests/unit/uncloseai-registry-refresh.test.ts— guards the new behaviour: a retired id is treated as dead and a not-yet-seen id still resolves to 65536.tests/unit/models-catalog-route.test.ts,tests/unit/combo-builder-options-route.test.ts— the three assertions reading the retired id / display name / root were red on this branch; they assert the live ones now.docs/reference/PROVIDER_REFERENCE.md— regenerated withnpm run gen:provider-reference; only the uncloseai row and the generated date lines move.Verification
node --import tsx/esm --test tests/unit/models-catalog-route.test.ts tests/unit/combo-builder-options-route.test.ts tests/unit/uncloseai-registry-refresh.test.tsnpx vitest run open-sse/services/__tests__/tierResolver.test.tsGET https://hermes.ai.unturf.com/v1/modelsturboderp/Qwen3.8-27B-exl3,rootthe same,max_model_len65536; the previously catalogued id is no longer servednpm run typecheck:corenpx eslint <changed files> --suppressions-location config/quality/eslint-suppressions.json(the CI invocation)The upstream read is the basis for calling this id live; because the roster is what rotates, the provider-level default is what stops the next rotation from repeating this.
Files
open-sse/config/providers/registry/uncloseai/index.tsopen-sse/config/freeModelCatalog.data.tssrc/shared/constants/providers/noauth.tstests/unit/uncloseai-registry-refresh.test.tstests/unit/models-catalog-route.test.tstests/unit/combo-builder-options-route.test.tsdocs/reference/PROVIDER_REFERENCE.mdrelease/v3.8.52, not from this change.