fix(api-server): honor chat completions request model - #25552
Open
lld1995 wants to merge 2 commits into
Open
Conversation
alt-glitch
added
type/bug
Collaborator
When a routing gateway (e.g. an alias like "high-accuracy" that fans out to "high-accuracy-m1/m2/m3") is configured as the inference backend, clients had no way to tell which concrete model actually served a given request — the response.model field always echoed the alias the client sent. Capture response.model from every successful upstream API call on AIAgent (_last_response_model) and surface it through run_conversation() so the api_server adapter can put the concrete backend id into the chat.completion response. Falls back to the client-supplied name when the upstream omits a model field, preserving the existing behavior for non-routing providers. Adds a regression test that verifies the alias is still used for provider selection (requested_model kwarg) while the response advertises the upstream-reported concrete backend.
This was referenced May 23, 2026
This was referenced May 23, 2026
Codename-11
added a commit
to Codename-11/hermes-agent
that referenced
this pull request
Jun 17, 2026
Desktop/external clients that send a model field in the session chat or
chat/stream request body can now override the gateway default for that
turn. _create_agent() and _run_agent() accept requested_model, which
takes priority over _resolve_gateway_model() when present.
Wired through:
- POST /api/sessions/{id}/chat
- POST /api/sessions/{id}/chat/stream
This enables the desktop composer model picker to hot-swap the model
mid-session without requiring a config.yaml write + gateway restart.
Upstream PRs NousResearch#22825, NousResearch#25552, and NousResearch#36110 address similar gaps on /v1/*
paths; this patch is scoped to the /api/sessions/* surface only.
teknium1
reviewed
Jul 13, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the API-server routing work. Current main now has a provider-aware request-model contract, but this PR still represents a different behavior rather than a redundant patch.
Problems
gateway/platforms/api_server.py:1299-1324resolves provider-specific runtime credentials for configuredmodel_routes. The PR's rawrequested_modelassignment atgateway/platforms/api_server.py:833changes only the model value, so it bypasses that provider/runtime resolution path.- Current main applies request model routing to chat completions, responses, and runs (
gateway/platforms/api_server.py:2194-2201,3285-3287,4285-4298); this PR covers only chat completions.
Suggested changes
- Define the contract for arbitrary request model strings versus configured
model_routes, including session/modelprecedence. - If arbitrary selection is intended, integrate it with provider credential/runtime resolution and cover all API-server endpoints and streaming paths.
Automated hermes-sweeper review.
| runtime_model = runtime_kwargs.pop("model", None) | ||
| model = requested_model or runtime_model or _resolve_gateway_model() | ||
|
|
||
| user_config = _load_gateway_config() |
Contributor
There was a problem hiding this comment.
Current main's model_routes path resolves provider-specific runtime credentials before changing the model (gateway/platforms/api_server.py:1299-1324). Selecting a raw request model here leaves the global provider/API key/base URL intact, so cross-provider model strings need a provider-resolution contract rather than only this precedence change.
teknium1
added
sweeper:risk-compatibility
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tests