fix(web): make plugin-provided web backends selectable - #52057
fix(web): make plugin-provided web backends selectable#52057ZOLAtheCodeX wants to merge 1 commit into
Conversation
`_is_backend_available()` gated backend selection on a hardcoded allowlist (exa/parallel/firecrawl/tavily/searxng/brave-free/ddgs/xai) and returned False for anything else. This silently broke the user / third-party web-provider plugin surface documented in agent/web_search_provider.py: a provider registered via `register_web_search_provider()` is discovered and dispatchable, but could never be *selected* through `web.search_backend` / `web.extract_backend` because this gate rejected its name, so selection fell through to the auto-detect cascade instead. Add a generic fallback: when the name isn't a built-in, defer to the web registry's own provider `is_available()`. Plugin discovery is triggered first because this gate can run before the dispatcher's `_ensure_web_plugins_loaded()`. No per-provider hardcoding — works for any registered plugin backend (verified with a Crawl4AI extract provider). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Duplicate of #36987 — that earlier-open PR adds the same |
|
Thanks for flagging — agreed, this duplicates #36987 and is covered more fully by #35327 (which also adds tests and fixes the related search-only dead-end). Closing in favor of those. One small thing that might be worth folding into whichever lands: this version calls |
Problem
agent/web_search_provider.pydocuments a user / third-party web-providerplugin surface: a provider subclasses
WebSearchProvider, registers viaPluginContext.register_web_search_provider(), and is selected throughweb.search_backend/web.extract_backend/web.backend.In practice that surface is only half-wired.
tools/web_tools.py::_is_backend_available()gates selection on a hardcoded allowlist:
Any backend name not in that list returns
False. So a plugin provider isdiscovered and dispatchable (the registry knows it), but can never be
selected:
_get_capability_backend()sees_is_backend_available(name)isFalse and falls through to the auto-detect cascade, picking some other
backend. For an extract-only plugin this surfaces as a misleading
"search-only backend" or "no extract provider configured" error even though
the provider is installed, enabled, and reports
is_available() == True.Fix
Add a generic fallback at the end of
_is_backend_available(): when the nameisn't one of the built-ins, defer to the web registry's own provider
is_available(). Plugin discovery is triggered first because this gate canrun before the dispatcher's own
_ensure_web_plugins_loaded()(in theextract path,
_get_extract_backend()is called before discovery).No per-provider hardcoding — it works for any backend registered through the
documented plugin path. Built-in backends are unaffected: they return from
their existing
if backend == ...branches before reaching the new code.Testing
Verified with a local Crawl4AI extract-only provider installed at
~/.hermes/plugins/web/crawl4ai/and enabled viaplugins.enabled:web.extract_backend: crawl4ai→_get_extract_backend()returnedsearxng(cascade fallback);web_extractfailed._get_extract_backend()returnscrawl4ai; a live extract returnsclean markdown. Built-in backends (firecrawl with a real key, searxng,
ddgs) continue to resolve and run unchanged.
Scope
One file, +13 lines, no behavior change for existing backends.
🤖 Generated with Claude Code