Skip to content

fix(aux): self-heal Nous-routed calls when a pinned model leaves the catalog - #37732

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-43b2e1f7
Jun 3, 2026
Merged

fix(aux): self-heal Nous-routed calls when a pinned model leaves the catalog#37732
teknium1 merged 1 commit into
mainfrom
hermes/hermes-43b2e1f7

Conversation

@teknium1

@teknium1 teknium1 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Nous-routed auxiliary calls now self-heal when a pinned model leaves the catalog, instead of 404'ing for the whole life of a long-running process.

Root cause: A long-lived process (gateway, watcher) caches the Nous Portal's recommended-models payload and can pin a model for its entire lifetime. When that model is later dropped from the Nous → OpenRouter catalog, every auxiliary call 404s with Model 'X' not found. The requested model does not exist in our configuration or OpenRouter catalog. until the process restarts. Observed in the wild with a vision_analyze call pinned to openai/gpt-5.4-mini.

Changes

  • agent/auxiliary_client.py:
    • _is_model_not_found_error() — 404/400 "not found / does not exist / not a valid model / catalog" predicate. Excludes billing keywords so it never overlaps _is_payment_error (free-tier/credit 404s stay with the payment path).
    • _refresh_nous_recommended_model() — force-refreshes the Portal recommendation and returns a model distinct from the one that just failed, else the known-good google/gemini-3-flash-preview default, else None.
    • Wired into both call_llm and async_call_llm error chains, right after the temperature/param retries. Gated to Nous-routed calls only (resolved_provider == "nous" or base_url host match) — generic aggregators are left alone to avoid masking real misconfig.
  • tests/agent/test_auxiliary_client.py: 12 new tests (TestIsModelNotFoundError, TestRefreshNousRecommendedModel) covering the incident error, edge cases, predicate non-overlap with payment, and all four self-heal branches.

Validation

Result
Predicate (incident error + 7 edge cases) correct; zero overlap with _is_payment_error
Self-heal vs live Portal openai/gpt-5.4-mini → stepfun/step-3.7-flash:free
tests/agent/test_auxiliary_client.py 203/203 (191 → 203, +12)

Infographic

stale-model-self-heal

@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
033bc4d 
…catalog

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: hermes/hermes-43b2e1f7 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9691 on HEAD, 9683 on base (🆕 +8)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5016 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/nous Nous Research API (OAuth) P2 Medium — degraded but workaround exists labels Jun 3, 2026
@teknium1
teknium1 merged commit ab2472e into main Jun 3, 2026
21 of 23 checks passed
@teknium1
teknium1 deleted the hermes/hermes-43b2e1f7 branch June 3, 2026 00:14
Yuki-14544869 pushed a commit to Yuki-14544869/hermes-agent that referenced this pull request Jun 4, 2026
@teknium1 @Yuki-14544869
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
c6c6a7d 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
davidgut1982 pushed a commit to davidgut1982/hermes-agent that referenced this pull request Jun 5, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
c4a879f 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
changman pushed a commit to changman/hermes-agent that referenced this pull request Jun 10, 2026
@teknium1 @changman
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
faa1bff 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
alt-glitch pushed a commit that referenced this pull request Jun 14, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
e3a2cc6 
…catalog (#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
kossteg pushed a commit to kossteg/hermes-agent that referenced this pull request Jun 16, 2026
@teknium1 @kossteg
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
311169a 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
8b89583 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
5794e4d 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
0a2d3e4 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
donbowman pushed a commit to donbowman/hermes-agent that referenced this pull request Jul 13, 2026
@teknium1 @donbowman
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
7f6d6d8 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
366e086 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
@teknium1
fix(aux): self-heal Nous-routed calls when a pinned model leaves the …
3fc83c6 
…catalog (NousResearch#37732)

A long-lived process (gateway, watcher) caches the Nous Portal's
recommended-models payload and can pin a model for its whole lifetime.
When that model is later dropped from the Nous -> OpenRouter catalog,
every auxiliary call 404s with 'model does not exist in our
configuration or OpenRouter catalog' until the process restarts.

Now such a 404 force-refreshes the Portal recommendation and retries
once with the current pick (or the gemini-3-flash-preview default).
Scoped to Nous-routed calls only.

- _is_model_not_found_error(): 404/400 'not found / does not exist /
  not a valid model' predicate, excludes billing keywords so it never
  overlaps _is_payment_error.
- _refresh_nous_recommended_model(): force-refresh fetch, returns a
  model distinct from the one that failed, else the known-good default.
- Wired into both call_llm and async_call_llm error chains.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/nous Nous Research API (OAuth) type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@teknium1 @alt-glitch