fix(providers): aux pool fallback, tool-use 404 classification, poolside wire types, codex text recovery (4-PR salvage) by teknium1 · Pull Request #58502 · NousResearch/hermes-agent

teknium1 · 2026-07-04T21:32:07Z

Summary

Four provider-boundary resilience fixes: anthropic auxiliary tasks no longer hard-fail when the credential pool has no usable entry, OpenRouter's tool-use 404 fast-fallbacks instead of burning retries, Poolside's integer wire types are coerced to strings, and the codex app-server no longer discards a completed answer when turn/completed never arrives.

Salvages #58474 (@Jigoooo), #58451 (@webtecnica), #58374 (@ostravajih), #58433 (@ooiuuii) onto current main, authorship preserved.

Changes

agent/auxiliary_client.py: _try_anthropic falls through to resolve_anthropic_token() when _select_pool_entry returns (True, None) — matching the fallback the openrouter and codex paths already have
agent/error_classifier.py: OpenRouter "No endpoints found that support tool use" 404 → model_not_found (retryable=False, should_fallback=True)
agent/transports/chat_completions.py + agent/chat_completion_helpers.py: coerce integer finish_reason / tool_call.id to strings (Poolside wire quirk)
agent/transports/codex_app_server_session.py: accept recorded final_text as terminal at the deadline when not interrupted and no error, instead of retiring the session with "turn timed out"
Follow-up commit drops unrelated package-lock.json churn (fix: classify OpenRouter 'no endpoints found that support tool use' 404 as model_not_found with fallback #58451) and the dead poolside picker entry (fix(poolside): handle integer finish_reason and tool_call id #58374 — no ProviderConfig/setup flow exists for it)

Validation

	Before	After
Anthropic pool select fails	aux tasks die: "no auxiliary client configured"	fall back to token resolver
OpenRouter tool-use 404	3-5 retries then generic error	immediate model fallback
Poolside `finish_reason: 24`	propagates as int, downstream misbehaves	`"24"`
Codex completes but no `turn/completed`	answer discarded, session retired	final text recovered

Targeted suites: auxiliary pool fallback regression, error_classifier, transports (incl. codex app-server), chat_completion_helpers — all pass.

Closes #58474. Closes #58451. Closes #58374. Closes #58433.

Infographic

webtecnica · 2026-07-04T22:34:59Z

Thanks for salvaging this 🙌 — appreciate you keeping authorship on #58451.

This salvage pack is cleaner than my standalone PR since it also fixes the Anthropic pool fallback, Poolside wire types, and Codex text recovery in one pass. LGTM — makes more sense to merge one consolidated PR than four individual ones for related provider-boundary issues.

…o usable entry _try_anthropic() hard-failed (return None, None) when the anthropic credential pool was present but had no selectable entry — e.g. the pooled OAuth token expired and its refresh_token had gone stale, so _select_pool_entry("anthropic") returned (True, None). This wedged every auxiliary task routed to Anthropic (goal judge surfaced "no auxiliary client configured") even when a perfectly valid ANTHROPIC_TOKEN / credentials-file token was available. The main session stayed healthy because it resolves the env token directly. The openrouter path (_try_openrouter) and codex path already fall through to their standalone credential on (True, None); anthropic was the only provider that hard-failed. Make _try_anthropic fall through to resolve_anthropic_token() on that branch so the three paths are symmetric: a temporarily dead pool entry must not block auxiliary tasks when a valid standalone credential exists. Adds a regression test covering: (1) pool present + no entry + valid env token -> client built from the env token, (2) pool present + no entry + no resolvable token -> clean (None, None), (3) base_url defaults correctly when falling through with pool_present=True.

…llback When OpenRouter routes to an endpoint that does not support tool/function calling, it returns HTTP 404 with the message 'No endpoints found that support tool use. Try disabling "browser_back".' The raw error body does not contain 'model not found' or any other _MODEL_NOT_FOUND_PATTERNS entry, so it falls through to FailoverReason.unknown with retryable=True. The retry loop wastes 3-5 attempts on the same deterministic rejection, then surfaces a confusing generic error instead of automatically failing over to a fallback model or provider. Adding the OpenRouter phrase to _MODEL_NOT_FOUND_PATTERNS classifies it as model_not_found (retryable=False, should_fallback=True), which triggers the client-error fast-fallback path in conversation_loop.py: the agent switches to a configured fallback model/provider before the user sees the error. Existing buffered guidance in conversation_loop.py (the 'support tool use' hint at line ~2967) remains intact and surfaces only if every fallback exhausts.

- ChatCompletionsTransport.normalize_response: convert integer finish_reason (e.g. 24) to string for Poolside compatibility - Chat completion helpers: handle integer tool_call.id during streaming by converting to string - Add Poolside as first-class CANONICAL_PROVIDERS entry (visible in CLI/TUI/desktop provider pickers)

- package-lock.json changes in #58451 were unrelated peer-flag churn - CANONICAL_PROVIDERS 'poolside' entry from #58374 has no ProviderConfig in hermes_cli/auth.py and no setup flow, so the picker entry would be dead; the wire-format coercions stand on their own

…n error quirks Adds a plugin seam at the top of agent/error_classifier.classify_api_error() (step 0, before the built-in pipeline) so model-provider plugins can classify their provider's error quirks without patching core: - New "classify_api_error" entry in VALID_HOOKS. Callbacks receive the parsed error context (provider, model, status_code, error_type, error_code, error_message, error_body, error, approx_tokens, context_length, num_messages), self-scope on `provider`, and return None to pass or a dict {"reason": "<FailoverReason name>", ...optional recovery-hint overrides}. - get_plugin_error_classification() helper mirrors get_pre_tool_call_block_message(): first valid result wins, invalid dicts and unknown reasons are skipped, callback exceptions are isolated — a broken plugin can never break classification. Zero behavior change when no plugin claims the error (all 179 existing classifier tests pass untouched). - Bundled reference plugin `openrouter-tool-use-404` (opt-in, like all bundled standalone plugins) re-implements PR NousResearch#58451: OpenRouter's "No endpoints found that support tool use" 404 carries no _MODEL_NOT_FOUND_PATTERNS signal, so it classifies as unknown/retryable and the retry loop burns 3-5 attempts on a deterministic rejection. The plugin classifies it as model_not_found (retryable=False, should_fallback=True) so the fast-fallback path fires immediately — demonstrating a waiting core PR converted to a publishable plugin. Motivation: ~10 open PRs are single-provider error-classification patches (NousResearch#58451, NousResearch#58355, NousResearch#58502, NousResearch#58474, NousResearch#58366, ...). This hook turns that whole class of contribution into plugin territory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FWMcB7RPSYUpsXDfBgwjzM

…n error quirks Adds a plugin seam at the top of agent/error_classifier.classify_api_error() (step 0, before the built-in pipeline) so model-provider plugins can classify their provider's error quirks without patching core: - New "classify_api_error" entry in VALID_HOOKS. Callbacks receive the parsed error context (provider, model, status_code, error_type, error_code, error_message, error_body, error, approx_tokens, context_length, num_messages), self-scope on `provider`, and return None to pass or a dict {"reason": "<FailoverReason name>", ...optional recovery-hint overrides}. - get_plugin_error_classification() helper mirrors get_pre_tool_call_block_message(): first valid result wins, invalid dicts and unknown reasons are skipped, callback exceptions are isolated — a broken plugin can never break classification. Zero behavior change when no plugin claims the error (all 179 existing classifier tests pass untouched). - Bundled reference plugin `openrouter-tool-use-404` (opt-in, like all bundled standalone plugins) re-implements PR #58451: OpenRouter's "No endpoints found that support tool use" 404 carries no _MODEL_NOT_FOUND_PATTERNS signal, so it classifies as unknown/retryable and the retry loop burns 3-5 attempts on a deterministic rejection. The plugin classifies it as model_not_found (retryable=False, should_fallback=True) so the fast-fallback path fires immediately — demonstrating a waiting core PR converted to a publishable plugin. Motivation: ~10 open PRs are single-provider error-classification patches (#58451, #58355, #58502, #58474, #58366, ...). This hook turns that whole class of contribution into plugin territory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FWMcB7RPSYUpsXDfBgwjzM

webdevtodayjason mentioned this pull request Jul 4, 2026

feat(plugins): transform_api_error_classification hook so provider plugins own their error quirks #58524

Closed

teknium1 force-pushed the salvage/provider-error-resilience branch from bbc4881 to af903f9 Compare July 4, 2026 22:28

Jigoooo and others added 6 commits July 4, 2026 15:39

fix(codex): recover final app-server text without completion

1bb0873

chore(release): AUTHOR_MAP entries for salvaged PR authors

08f4f5b

teknium1 force-pushed the salvage/provider-error-resilience branch from af903f9 to 08f4f5b Compare July 4, 2026 22:40

teknium1 enabled auto-merge (rebase) July 4, 2026 22:40

teknium1 merged commit 10f7cb0 into main Jul 4, 2026
29 checks passed

teknium1 deleted the salvage/provider-error-resilience branch July 4, 2026 22:44

briandevans mentioned this pull request Jul 4, 2026

fix(chat-completions): coerce integer tool_call id in non-streaming normalize_response #58562

Open

13 tasks

Navigation Menu

Uh oh!

fix(providers): aux pool fallback, tool-use 404 classification, poolside wire types, codex text recovery (4-PR salvage) - #58502

fix(providers): aux pool fallback, tool-use 404 classification, poolside wire types, codex text recovery (4-PR salvage)#58502
teknium1 merged 6 commits into
mainfrom
salvage/provider-error-resilience

teknium1 commented Jul 4, 2026

Uh oh!

webtecnica commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

teknium1 commented Jul 4, 2026

Summary

Changes

Validation

Infographic

Uh oh!

webtecnica commented Jul 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants