feat(web): multi-source fallback chain and search_engine parameter by Icather · Pull Request #53151 · NousResearch/hermes-agent

Icather · 2026-06-26T16:11:38Z

What does this PR do?

When a search provider fails, Hermes previously returned an error immediately — no retry with another backend. This adds a configurable fallback chain and a search_engine parameter so the model can pick providers and Hermes retries on failure.

Depends on: #53149 (add 9 new providers)

Type of Change

New feature (non-breaking addition)

Related Issue

Follows the intent of #35690 (closed without review) and extends it with dynamic engine names from the registry.

Changes Made

web_search_tool(query, limit, search_engine="auto") — new search_engine parameter; "auto" walks the fallback chain, explicit names run a single backend
_get_fallback_chain() — reads web.fallback_backends from config, appends auto-detected providers
_search_with_fallback() — walks chain, skips unavailable/error/0-results backends, stops at first success
_get_valid_engine_names() — dynamically sources valid engine names from list_provider_names(), so adding a new provider (PR 1) automatically extends the enum

How to Test

Merge feat(web): add 9 search providers — serper, baidu, bocha, qiniu-baidu, serpapi, jina, google-cse, sogou, 360-search #53149 (provider PR) first
Set BRAVE_SEARCH_API_KEY and SERPER_API_KEY
Configure web.fallback_backends: [brave-free, serper, ddgs]
Call web_search — should try brave-free first; unset the key to verify fallback

Checklist

My commit messages follow Conventional Commits
I searched for existing PRs (fix: restore search fallback routing #35690)
My PR contains only changes related to this feature
I have tested on my platform: Windows 10
I have added tests for my changes

…u-baidu, serpapi, jina, google-cse, sogou, 360-search) v0.17 removed the Parallel free MCP fallback (NousResearch#46350), sharply reducing free search options. This adds 9 new providers as plugins following the existing brave-free/ddgs pattern, registered in backend_candidates in composite quality-score order after the existing backends. New functional providers (search-only): - serper — Google SERP, 2,500 free queries, no credit card - baidu — Baidu AI Search, 100/day free, native Chinese content - bocha — Chinese market leader, 1,000 free starter queries - qiniu-baidu — Qiniu Cloud Baidu, 3M tokens for new users - serpapi — Multi-engine, 100/month free - jina — Full-page extraction, 10M tokens free (blocked in China) - google-cse — Google Custom Search, 100/day (blocked in China) Registry-only entries (no public API, listed for completeness): - sogou, 360-search Two prior attempts (NousResearch#41015 Serper by ViezeVingertjes, NousResearch#35690 fallback by jonathanwxh-cell) were self-closed without review. This salages their approach with a complete provider surface covering both Western and Chinese search engines. Changes: - plugins/web/*/ — 27 new files (provider.py, plugin.yaml, __init__.py per backend) - tools/web_tools.py — +9 entries in _KNOWN_WEB_BACKENDS set, +9 in backend_candidates tuple, +_check_provider_available() helper, generic plugin probe in _is_backend_available()

…r for web_search_tool When a search provider fails, Hermes previously returned an error immediately — no retry with another backend. This adds: - search_engine parameter to web_search_tool() — model can specify "baidu", "serper", "ddgs", or "auto" to walk the fallback chain. Valid values are dynamically sourced from list_provider_names(), so adding a new provider automatically extends the enum — zero changes. - web.fallback_backends config key — user-defined ordered list of providers to try. If unset, all registered providers are tried in registration order (paid first, free last). - _search_with_fallback() — walks the chain, skipping backends that are not found, not available, return errors, or return 0 results. Stops at first success. Returns error trace on total failure. This PR depends on feat/add-free-web-search-providers (PR NousResearch#53149) which adds the 9 new providers that this chain can fall back through.

Copilot

Pull request overview

This PR extends Hermes’ web search tool to support multi-backend operation by introducing an "auto" fallback chain mode and a search_engine parameter intended to let the model (or config) select a specific provider, while also adding several new plugin-based search providers.

Changes:

Adds fallback-chain logic (_get_fallback_chain, _search_with_fallback) and search_engine="auto" support in web_search_tool().
Extends backend availability probing to include plugin-registered providers.
Introduces new plugins/web/* provider implementations (Baidu/Bocha/Serper/SerpAPI/Jina/Google CSE/Sogou/360-search/Qiniu-Baidu).

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 23 comments.

Show a summary per file

File	Description
tools/web_tools.py	Adds fallback-chain search dispatch + generic plugin availability probing, and extends backend lists.
plugins/web/sogou/provider.py	Adds an “unavailable” registry-only Sogou provider implementation.
plugins/web/sogou/plugin.yaml	Declares the Sogou plugin manifest.
plugins/web/sogou/init.py	Adds Sogou plugin package stub (needs `register(ctx)`).
plugins/web/serper/provider.py	Adds Serper provider implementation.
plugins/web/serper/plugin.yaml	Declares the Serper plugin manifest.
plugins/web/serper/init.py	Adds Serper plugin package stub (needs `register(ctx)`).
plugins/web/serpapi/provider.py	Adds SerpAPI provider implementation.
plugins/web/serpapi/plugin.yaml	Declares the SerpAPI plugin manifest.
plugins/web/serpapi/init.py	Adds SerpAPI plugin package stub (needs `register(ctx)`).
plugins/web/qiniu-baidu/provider.py	Adds Qiniu-Baidu provider implementation.
plugins/web/qiniu-baidu/plugin.yaml	Declares the Qiniu-Baidu plugin manifest.
plugins/web/qiniu-baidu/init.py	Adds Qiniu-Baidu plugin package stub (needs `register(ctx)` and relative import due to hyphenated dir).
plugins/web/jina/provider.py	Adds Jina provider implementation.
plugins/web/jina/plugin.yaml	Declares the Jina plugin manifest.
plugins/web/jina/init.py	Adds Jina plugin package stub (needs `register(ctx)`).
plugins/web/google-cse/provider.py	Adds Google CSE provider implementation.
plugins/web/google-cse/plugin.yaml	Declares the Google CSE plugin manifest.
plugins/web/google-cse/init.py	Adds Google CSE plugin package stub (needs `register(ctx)` and relative import due to hyphenated dir).
plugins/web/bocha/provider.py	Adds Bocha provider implementation.
plugins/web/bocha/plugin.yaml	Declares the Bocha plugin manifest.
plugins/web/bocha/init.py	Adds Bocha plugin package stub (needs `register(ctx)`).
plugins/web/baidu/provider.py	Adds Baidu provider implementation.
plugins/web/baidu/plugin.yaml	Declares the Baidu plugin manifest.
plugins/web/baidu/init.py	Adds Baidu plugin package stub (needs `register(ctx)`).
plugins/web/_360_search/provider.py	Adds an “unavailable” registry-only 360-search provider implementation.
plugins/web/_360_search/plugin.yaml	Declares the 360-search plugin manifest.
plugins/web/_360_search/init.py	Adds 360-search plugin package stub (needs `register(ctx)`).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    try:
+        from agent.web_search_registry import list_provider_names
+        return set(list_provider_names())
+    except Exception:
+        return set()


+    primary = (cfg.get("backend") or "").lower().strip()
+    if primary:
+        chain.append(primary)
+
+    user_fbs = cfg.get("fallback_backends", [])
+    if isinstance(user_fbs, str):
+        user_fbs = [b.strip() for b in user_fbs.split(",") if b.strip()]
+    for b in user_fbs:
+        if b not in chain:
+            chain.append(b)
+
+    # Append remaining registered providers (those not already listed).
+    # The registry returns providers in registration order, which for
+    # bundled plugins is the backend_candidates tuple order.
+    try:
+        from agent.web_search_registry import list_provider_names
+        for name in list_provider_names():
+            if name not in chain:
+                chain.append(name)
+    except Exception:
+        pass


+            # Attach fallback trace for observability
+            response_data.setdefault("_fallback_trace", errors)


+def web_search_tool(query: str, limit: int = 5, search_engine: str = "auto") -> str:
    """
-    Search the web for information using available search API backend.
+    Search the web for information using available search API backends.

    This function provides a generic interface for web search that can work
-    with multiple backends (Parallel or Firecrawl).
+    with multiple backends. When ``search_engine`` is ``"auto"`` (default),
+    backends from ``web.fallback_backends`` are tried in order until one
+    succeeds.  Explicit engine names run a single backend with no fallback.


+def web_search_tool(query: str, limit: int = 5, search_engine: str = "auto") -> str:
    """
-    Search the web for information using available search API backend.
+    Search the web for information using available search API backends.

    This function provides a generic interface for web search that can work
-    with multiple backends (Parallel or Firecrawl).
+    with multiple backends. When ``search_engine`` is ``"auto"`` (default),
+    backends from ``web.fallback_backends`` are tried in order until one
+    succeeds.  Explicit engine names run a single backend with no fallback.


+        except Exception as e:
+            return {"success": False, "error": str(e)}


+        except Exception as e:
+            return {"success": False, "error": str(e)}


+        except Exception as e:
+            return {"success": False, "error": str(e)}


+import logging
+import os
+from typing import Any, Dict
+
+from agent.web_search_provider import WebSearchProvider
+
+logger = logging.getLogger(__name__)


+import logging
+import os
+from typing import Any, Dict
+
+from agent.web_search_provider import WebSearchProvider
+
+logger = logging.getLogger(__name__)


Icather added 2 commits June 27, 2026 00:03

Copilot AI review requested due to automatic review settings June 26, 2026 16:11

Copilot started reviewing on behalf of Icather June 26, 2026 16:12 View session

Icather mentioned this pull request Jun 26, 2026

feat(tools): hermes tools web reorder — interactive search provider priority #53152

Closed

6 tasks

Icather marked this pull request as draft June 26, 2026 16:16

Icather closed this Jun 26, 2026

Copilot AI reviewed Jun 26, 2026

View reviewed changes

alt-glitch added type/feature New feature or request comp/tools Tool registry, model_tools, toolsets tool/web Web search and extraction P3 Low — cosmetic, nice to have labels Jun 26, 2026

GottZ mentioned this pull request Jul 30, 2026

feat(web): multi-source fallback chain and search_engine parameter #53158

Open

8 tasks

Navigation Menu

Uh oh!

feat(web): multi-source fallback chain and search_engine parameter - #53151

feat(web): multi-source fallback chain and search_engine parameter#53151
Icather wants to merge 2 commits into
NousResearch:mainfrom
Icather:feat/web-search-fallback-chain

Icather commented Jun 26, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		# Attach fallback trace for observability
		response_data.setdefault("_fallback_trace", errors)

		except Exception as e:
		return {"success": False, "error": str(e)}

Uh oh!

Conversation

Icather commented Jun 26, 2026

What does this PR do?

Type of Change

Related Issue

Changes Made

How to Test

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants