Skip to content

fix(config): resolve direct model aliases at runtime - #19213

Open
LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix/custom-provider-model-aliases-18954
Open

LeonSGP43 wants to merge 1 commit into
NousResearch:mainfrom
LeonSGP43:fix/custom-provider-model-aliases-18954

Conversation

@LeonSGP43

Copy link
Copy Markdown
Contributor

Summary

Fixes #18954.

Direct model_aliases: entries were resolved in the /model switch flow, but normal runtime startup could still pass the alias itself as the API model. That broke custom-provider aliases such as opus -> claude-opus-4-6, especially when the provider was already explicit or came from config/env.

This PR adds a small direct-alias lookup helper and applies it before runtime provider construction in:

  • interactive CLI credential resolution
  • oneshot hermes chat -m ... runtime setup, including explicit --provider
  • gateway session runtime setup from config defaults

The change keeps the existing /model behavior intact and preserves direct-alias base_url routing for custom endpoints.

Verification

  • scripts/run_tests.sh tests/cli/test_cli_provider_resolution.py::test_runtime_resolution_resolves_direct_model_alias tests/gateway/test_session_model_override_routing.py::test_gateway_runtime_resolves_direct_model_alias tests/hermes_cli/test_oneshot_model_aliases.py::test_oneshot_resolves_direct_model_alias_with_explicit_provider
    • 3 passed, 3 warnings
  • scripts/run_tests.sh tests/hermes_cli/test_ollama_cloud_auth.py tests/gateway/test_session_model_override_routing.py tests/hermes_cli/test_oneshot_model_aliases.py
    • 35 passed, 4 warnings
  • scripts/run_tests.sh tests/cli/test_cli_provider_resolution.py
    • 20 passed, 4 warnings

@alt-glitch alt-glitch added P2 Medium — degraded but workaround exists type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles labels May 3, 2026
@StartupBros

Copy link
Copy Markdown
Contributor

Hey @LeonSGP43 — the duplicate-detector bot linked this PR alongside #29198, which I opened for the same issue. Wanted to share the architectural difference in case it's useful for rescoping.

#29198 fixes only the hermes chat dispatch path (HermesCLI._init_agent in cli.py) — narrow surface, two helpers (_resolve_direct_alias_for_chat, _apply_resolved_alias), 5 regression tests in tests/hermes_cli/test_regression_16767.py. It addresses #16767 and #18954 specifically.

This PR is broader: it adds resolve_direct_alias() to model_switch.py as a proper API and threads it through three callers (cli.py::_ensure_runtime_credentials, gateway/run.py::_resolve_session_agent_runtime, hermes_cli/oneshot.py::_run_agent). The resolve_runtime_provider(target_model=...) signature change is a wider blast radius but solves the alias problem at every entry point.

If #29198 lands first, this PR could rescope to just the gateway + oneshot callers (the parts #29198 doesn't cover) — that'd keep the broader architectural improvement landable while reducing conflict surface. The resolve_direct_alias() API on model_switch is genuinely cleaner than my inline helper and worth keeping regardless. Happy to coordinate if you'd like to consolidate.

@StartupBros

Copy link
Copy Markdown
Contributor

Update on my earlier comment: I closed #29198 in favor of this PR. After tracing the actual call flow on current main:

  • HermesCLI._init_agent calls _ensure_runtime_credentials() first (cli.py:4502)
  • Both real _init_agent callers (cli.py:11017, cli.py:14442) pass model_override=turn_route["model"], and _resolve_turn_agent_config sets turn_route["model"] = self.model with no transformation

So once this PR's _ensure_runtime_credentials patch resolves self.model, every downstream path (chat, gateway, oneshot, credentials) sees the canonical model. #29198 was a strictly-narrower subset and is now closed.

Validation against current main (May 20):

File Status
hermes_cli/model_switch.py (new resolve_direct_alias API) applies cleanly
cli.py (_ensure_runtime_credentials patch) applies cleanly
hermes_cli/oneshot.py (_run_agent patch) applies cleanly
gateway/run.py (_resolve_session_agent_runtime patch) conflicts — that function has drifted on main since this PR was opened

With the 3 cleanly-applicable hunks + this PR's tests applied to current main, the full alias-resolution test surface passes:

  • tests/cli/test_cli_provider_resolution.py — passes including the new test_runtime_resolution_resolves_direct_model_alias
  • tests/hermes_cli/test_oneshot_model_aliases.py — new file, passes
  • tests/hermes_cli/test_regression_16767.py and tests/hermes_cli/test_ollama_cloud_auth.py — unaffected, still green

55 tests pass.

If you have bandwidth to rebase, the only material work is reconciling the gateway/run.py hunk against the current _resolve_session_agent_runtime (the function moved + grew but the integration point should still be there). Happy to take a pass at that rebase + open a PR against your branch if useful — just say the word.

@xiaoyaner0201

Copy link
Copy Markdown

I hit this same bug in a real config where the default provider is openai-codex and model_aliases points an alias like opus47 at a named custom provider.

Observed failure before the fix:

hermes chat -q 'Reply exactly OK' -m opus47 -Q --toolsets safe

The raw alias was sent to the default Codex provider instead of being resolved to the alias' custom provider/model, producing a provider/model mismatch. With an explicit custom provider, the provider was correct but the raw alias could still reach the upstream if not expanded early enough.

I checked this PR because it appears to cover the exact runtime/startup alias path. The targeted tests pass locally for me on the PR branch:

python -m pytest \
  tests/hermes_cli/test_oneshot_model_aliases.py \
  tests/cli/test_cli_provider_resolution.py \
  tests/gateway/test_session_model_override_routing.py \
  -q -o 'addopts='

24 passed

This matches the behavior I needed: -m <direct-alias> is resolved before the default provider can claim it, and explicit-provider paths still expand the alias model before dispatch.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for isolating the runtime alias gap. The premise remains valid on current main: explicit-provider oneshot skips direct-alias lookup at hermes_cli/oneshot.py:336, then passes the raw alias as target_model at hermes_cli/oneshot.py:369-373; CLI and gateway similarly retain the raw configured model (hermes_cli/cli_agent_setup_mixin.py:41-45, gateway/run.py:3758-3796).

Problems

  • The same startup bug remains in the TUI: _resolve_startup_runtime() returns the configured model unchanged at tui_gateway/server.py:2180-2212, and _make_agent() resolves it without alias expansion at tui_gateway/server.py:4567-4575. The PR does not cover that sibling path.
  • The new tests mock both alias lookup and runtime resolution. For this config-propagation change, add a temp-HERMES_HOME integration test using a real model_aliases entry, consistent with AGENTS.md:84-87.

Suggested changes

  • Transplant the CLI logic into the current hermes_cli/cli_agent_setup_mixin.py::_ensure_runtime_credentials and reconcile the gateway integration at current gateway/run.py::_resolve_session_agent_runtime.
  • Apply the same shared normalization to TUI startup, while retaining explicit-provider precedence where applicable.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
@alt-glitch alt-glitch added the needs-decision Awaiting maintainer decision before any implementation label Jul 18, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related to open #30716 and #62534: this branch covers broader CLI/oneshot/gateway runtime setup, while those target narrower CLI choke points. Current maintainer review identifies an unaddressed TUI startup path, so this needs consolidation rather than duplicate closure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model aliases not resolved for custom providers

5 participants