Skip to content

feat(web): delegate backend availability check to plugin registry for unknown backends - #56761

Closed
Josh-OK wants to merge 1 commit into
NousResearch:mainfrom
Josh-OK:feat/plugin-aware-backend-availability
Closed

Josh-OK wants to merge 1 commit into
NousResearch:mainfrom
Josh-OK:feat/plugin-aware-backend-availability

Conversation

@Josh-OK

@Josh-OK Josh-OK commented Jul 2, 2026

Copy link
Copy Markdown

Problem

_is_backend_available() in web_tools.py is a hardcoded whitelist of backend names. Any plugin-registered backend (e.g., crawl4ai, future extract-only providers) is silently rejected because the function doesn't recognize its name — even though the plugin has already registered via PluginContext.register_web_search_provider() with its own is_available() check.

This means setting web.extract_backend: crawl4ai in config has no effect unless someone first patches _is_backend_available() to add a per-backend block. Every new extract-only plugin would need its own hardcoded entry.

Fix

Add a fallthrough at the bottom of _is_backend_available(): when the backend name doesn't match any hardcoded entry, ask the provider registry whether a plugin has registered a provider for that name, and delegate to provider.is_available().

Why this approach

  • One change, all plugins: No per-backend special cases in core. Any plugin that registers via register_web_search_provider() works automatically.
  • Plugin owns its check: The plugin's own is_available() (which it implements via the WebSearchProvider ABC) is the authority — core doesn't need to know about specific packages or env vars.
  • Matches existing pattern: ddgs already checks package importability. This is the same idea, generalized to the plugin registry.
  • Minimal surface: 11 lines, no new imports at module level, lazy import prevents circular dependency.

Tested

Verified end-to-end on a host with the crawl4ai plugin registered. Backward-compatible: the hardcoded checks run first and are unchanged. The fallthrough only fires for names not already recognized.

… unknown backends

_is_backend_available() is a hardcoded whitelist of backend names.
Any plugin-registered backend (crawl4ai, future extract-only providers)
is silently rejected because the function doesn't recognize its name.

Add a fallthrough check: when the backend name doesn't match any
hardcoded entry, ask the provider registry if a plugin has registered
a provider for that name, and delegate to provider.is_available().
This makes the gatekeeper plugin-aware without adding per-backend
special cases — one change covers every current and future plugin.
@alt-glitch alt-glitch added type/feature New feature or request comp/tools Tool registry, model_tools, toolsets tool/web Web search and extraction comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Jul 2, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Duplicate of #36987 — both add the same plugin-registry fallthrough (get_provider(backend).is_available()) at the same single call site in _is_backend_available(). #36987 is the earlier/canonical PR. This gate has a saturated competing-fix cluster (#36987, #56201, #32465, #35327 …) against anchor issue #31873; #56201 is the broader 3-site superset.

@kshitijk4poor

Copy link
Copy Markdown
Contributor

Superseded by #57779, which fixes this same bug class (plugin-registered web providers invisible to the tool-availability gate) as a single chokepoint in _is_backend_available() on current main — cascading to _get_backend(), _get_capability_backend(), and check_web_api_key().

Your PR targeted the same gate and informed the approach — thanks for the contribution, credited in the merged PR body. Closing as superseded. #57779

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have tool/web Web search and extraction type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants