Skip to content

feat: advertise configured governed API server models - #36110

Open
bshenderson wants to merge 1 commit into
NousResearch:mainfrom
bshenderson:feat/api-server-governed-model-routing
Open

feat: advertise configured governed API server models#36110
bshenderson wants to merge 1 commit into
NousResearch:mainfrom
bshenderson:feat/api-server-governed-model-routing

Conversation

@bshenderson

@bshenderson bshenderson commented May 31, 2026

Copy link
Copy Markdown

Summary

  • load API server provider/model truth from Hermes config instead of advertising a static hermes-agent model only
  • advertise governed configured-provider models on /v1/models
  • resolve request-time model routing from the same configured provider/model catalog for session chat, chat completions, responses, and runs
  • add focused tests covering governed model advertisement and explicit/default/named-profile model resolution

Why

Today the API server can expose a misleading model surface: the advertised model list and request-time routing can drift from the provider/model truth actually configured in Hermes. That makes governed Open WebUI and other API clients bind against an authority surface that is not the real runtime authority.

This change makes the API server expose and resolve models from the same configured source of truth.

Scope

  • gateway/platforms/api_server.py
  • tests/gateway/test_api_server.py

Behavior change

  • /v1/models now includes configured-provider model IDs derived from Hermes config
  • request-time model resolution now uses the configured provider/model catalog
  • explicit requested model IDs remain supported when present in configured provider truth
  • default and named-profile resolution paths are covered by focused tests

Validation

  • python3 -m py_compile gateway/platforms/api_server.py tests/gateway/test_api_server.py
  • /opt/hermes/.venv/bin/python -m pytest -q tests/gateway/test_api_server.py -k 'TestModelsEndpoint and (models_returns_hermes_agent or models_advertise_governed_provider_models_from_config or models_returns_profile_name or resolve_model_name_explicit or resolve_model_name_default_profile or resolve_model_name_named_profile)'

Result:

  • 6 passed

Risks / reviewer focus

  • compatibility expectations for clients that assumed only the synthetic hermes-agent model ID
  • config parsing and fallback behavior when providers or profiles are incomplete
  • consistency between advertised model IDs and downstream request-routing normalization

Notes

  • Branch lives in the real fork bshenderson/hermes-agent.
  • Change commit on the fork branch: 778740b78017c142cc76ff61663abf808257199f.

@bshenderson

Copy link
Copy Markdown
Author

Maintainer validation summary:

  • Branch is cleanly based on current main and is ahead by 1 commit only.
  • Diff scope is limited to 2 files:
    • gateway/platforms/api_server.py
    • tests/gateway/test_api_server.py
  • Local validation run before PR:
    • python3 -m py_compile gateway/platforms/api_server.py tests/gateway/test_api_server.py
    • /opt/hermes/.venv/bin/python -m pytest -q tests/gateway/test_api_server.py -k 'TestModelsEndpoint and (models_returns_hermes_agent or models_advertise_governed_provider_models_from_config or models_returns_profile_name or resolve_model_name_explicit or resolve_model_name_default_profile or resolve_model_name_named_profile)'
  • Result: 6 passed

Reviewer focus:

  • backward compatibility for clients that assumed only the synthetic hermes-agent model ID
  • consistency between /v1/models advertisement and request-time model resolution
  • fallback behavior when configured providers/profiles are incomplete

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery labels May 31, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to the existing /v1/models feature chain: #24946 (open), #23068 (closed), #26667 (closed dup), #30311 (closed dup), and original request #4405. This PR takes a different approach (governed model routing from config) compared to the provider-prefixed IDs in prior attempts.

@mxnstrexgl mxnstrexgl left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — automated review passed. No security, quality, or test coverage issues detected.

Codename-11 added a commit to Codename-11/hermes-agent that referenced this pull request Jun 17, 2026
Desktop/external clients that send a model field in the session chat or
chat/stream request body can now override the gateway default for that
turn. _create_agent() and _run_agent() accept requested_model, which
takes priority over _resolve_gateway_model() when present.

Wired through:
- POST /api/sessions/{id}/chat
- POST /api/sessions/{id}/chat/stream

This enables the desktop composer model picker to hot-swap the model
mid-session without requiring a config.yaml write + gateway restart.

Upstream PRs NousResearch#22825, NousResearch#25552, and NousResearch#36110 address similar gaps on /v1/*
paths; this patch is scoped to the /api/sessions/* surface only.
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused configured-provider proposal. The automatic catalog behavior is still distinct from current main, so it remains worth preserving for salvage.

Problems

  • Current main introduced model_routes after this branch: gateway/platforms/api_server.py:1191 parses explicit aliases, :1461 advertises them, and the chat/Responses/runs handlers resolve and pass them to agent creation (:2231, :3352, :4336). The PR predates that routing contract; GitHub currently marks it dirty. Its parallel _configured_model_routes path needs to compose with model_routes, rather than replace the surrounding routing implementation.
  • The added test verifies listing and internal route lookup, but does not prove that any HTTP endpoint constructs an agent with the configured provider/model. Current route tests demonstrate that endpoint-level pattern at tests/gateway/test_api_server.py:3968.

Suggested changes

  • Rework the catalog-derived entries into the existing model_routes resolver while preserving explicit alias routing and precedence.
  • Add endpoint-level provider/model-resolution coverage for each intended API surface.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 13, 2026
@alt-glitch alt-glitch removed the sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data label Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants