fix(web): load user extract plugins before backend resolution

[Plauschi]

Summary

• discover configured non-legacy web plugins before probing backend availability
• prevent a fresh web_extract process from falling back to search-only ddgs when web.extract_backend names a user plugin
• classify nested per-result "error": null as success while preserving meaningful nested error messages

Reproduction

With web.backend=ddgs, web.extract_backend=jina-reader, and the user plugin enabled, a fresh process selected ddgs before plugin discovery and returned a search-only backend error. After extraction succeeded, the display detector also marked the result as failed because the response contained a nested "error": null key.

Test plan

• 162 passed across the focused web-provider and display suites
• real fresh-process extraction of https://example.com through the user jina-reader plugin
• gateway API smoke returned GATEWAY_EXTRACT_OK and logged tool web_extract completed

[teknium1]

Thanks for isolating the selection-time ordering bug. Current main confirms the premise: web_extract_tool() resolves _get_extract_backend() at tools/web_tools.py:857, while its existing discovery call follows at line 866; _get_capability_backend() therefore probes a cold plugin registry at line 306. The display change also fixes the current generic "error"-key heuristic at agent/display.py:1311 for nested error: null envelopes.

Problems

• check_web_api_key() is a sibling cold-registry gate: it probes configured and active providers at tools/web_tools.py:1062-1081 without discovery, while both web tools use it as check_fn at lines 1218 and 1232. A custom extract-only provider can still be omitted before this dispatcher-level fix runs.

Suggested changes

• Call the existing _ensure_web_plugins_loaded() before the availability probes in check_web_api_key() and cover that path with a cold-registry custom extract-provider test.

Automated hermes-sweeper review.

[teknium1]

Please apply this same idempotent discovery guard to check_web_api_key(). That function remains a pre-dispatch check_fn for both web tools and currently probes custom-provider availability against a cold registry, so an extract-only custom provider can still leave the tools unavailable before this corrected selector runs.

[GottZ]

This was generated by AI during triage.

Summary

Twenty-four PRs address three linked causes: cold-registry discovery ordering, hardcoded provider-selection/tool gates, and non-actionable web-tool failures. #34563 and #57779 are the merged reference implementations for dispatcher discovery and registry-aware selection, while #67110, #67309, #71791, and #72646 address the remaining selection-before-discovery gap and #58359 separately addresses failure guidance.

Related pull requests

• fix(web): ensure plugin discovery before web_*_tool registry lookups #27584 [closed] best fix — (+282/-0) — n/a: Adds discovery before search, extract, and crawl registry lookups; it remains relevant because its live search/extract changes were salvaged into merged fix(web): ensure plugin discovery before web_*_tool registry lookups #34563, the recorded best-fix reference.
• fix(web_tools): trigger plugin discovery before registry dispatch (#27683) #27700 [closed] best fix — (+129/-0) — n/a: Implements the same pre-dispatch discovery invariant as fix(web): ensure plugin discovery before web_*_tool registry lookups #27584 with focused tests; it was closed as a duplicate before fix(web): ensure plugin discovery before web_*_tool registry lookups #34563 landed the maintained implementation.
• fix(web): discover provider plugins before web dispatch #28202 [closed] fixes — (+48/-0) — n/a: Discovers provider plugins before web dispatch, duplicating fix(web): ensure plugin discovery before web_*_tool registry lookups #27584/fix(web_tools): trigger plugin discovery before registry dispatch (#27683) #27700; it was closed in favor of those earlier fixes and superseded by fix(web): ensure plugin discovery before web_*_tool registry lookups #34563.
• fix(web_tools): delegate backend availability to provider registry is_available() #28652 [closed] fixes — (+161/-28) — n/a: Makes backend selection, availability, and the tool gate registry-aware; despite its keep_open review identifying missing gate coverage, the amended tests and approach were salvaged into merged fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(web_tools): delegate _is_backend_available to provider registry for plugin backends #31829 [closed] fixes — (+11/-0) — n/a: Delegates unknown backend availability to provider.is_available() but leaves broader selection and gate paths unchanged; it was closed as superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(web): honor plugin-registered search providers in availability gate #31887 [closed] fixes — (+86/-6) — n/a: Makes check_web_api_key() recognize an active plugin provider, but its visible diff leaves configured-provider routing able to choose another backend; it was closed as superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• [codex] fix(web): honor registered provider availability #32465 [closed] fixes — (+117/-5) — n/a: Covers shared and per-capability custom-provider selection plus the tool gate with tests; it was closed because fix(web_tools): honor plugin-registered provider availability at the tool gate #57779 implemented this issue class on main.
• fix(tools): check registered web search plugins in check_web_api_key #33516 [closed] fixes — (+99/-0) — n/a: Adds configured-provider lookup to check_web_api_key(); it was closed after the broader shared-path implementation landed through fix(web_tools): honor plugin-registered provider availability at the tool gate #57779 and 0a9d42c.
• feat: allow web tools to discover plugin-registered backends #33902 [closed] partial — (+9/-1) — n/a: Treats registry presence as availability without calling provider.is_available(), making it a narrower fix(web_tools): delegate _is_backend_available to provider registry for plugin backends #31829 duplicate; it was closed after fix(web_tools): honor plugin-registered provider availability at the tool gate #57779 supplied the tested implementation.
• fix(web): ensure plugin discovery before web_*_tool registry lookups #34563 [merged] best fix — (+212/-0) — n/a: Merged reference implementation salvaged from fix(web): ensure plugin discovery before web_*_tool registry lookups #27584, loading plugins before the two live web dispatchers and directly fixing the cold-registry failure in web_extract fails with "No web extract provider configured" despite extract_backend: firecrawl and valid API key #27580/web_tools.py: missing _ensure_plugins_discovered() at search/extract/crawl dispatch sites causes web tools to silently fail #27683.
• fix(tools): honor available plugin extract/search backends in web selection #35327 [closed] best fix — (+213/-12) — n/a: Honors plugin capability backends and falls through from a search-only provider to an active extract provider; it remains the recorded best-fix reference for [Bug]: web_extract gives dead-end error with only SearXNG configured #32698 although closed as superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779 for routing.
• fix(web): dispatch plugin-registered backends instead of falling through to DDGS #36094 [closed] best fix — (+119/-0) — n/a: Accepts configured registry providers and delegates their availability, preventing custom names from falling through to DDGS; it remains the recorded best-fix reference for [Bug]: Custom web search backends silently fall through to DDGS due to hardcoded allowlist #35970 although superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(web): honor plugin-registered backend availability #38375 [closed] fixes — (+53/-0) — n/a: Loads plugins and delegates unknown-name availability to provider.is_available() for custom capability backends; it was closed as superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(web): make plugin-provided web backends selectable #52057 [closed] fixes — (+13/-0) — n/a: Adds discovery plus registry fallback inside _is_backend_available(); the author closed it in favor of earlier competing fixes later consolidated by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• feat(web): allow plugin-registered backends through the availability gate #56201 [closed] fixes — (+29/-0) — n/a: Updates _get_backend(), _is_backend_available(), and check_web_api_key() for registered providers; it was closed as already implemented more completely by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• feat(web): delegate backend availability check to plugin registry for unknown backends #56761 [closed] fixes — (+11/-0) — n/a: Adds the same unknown-backend provider.is_available() fallback as fix(web_tools): delegate _is_backend_available to provider registry for plugin backends #31829; it was closed as superseded by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(tools): honor plugin web providers in extract/search dispatch #57704 [closed] best fix — (+232/-50) — n/a: Directly honors configured plugin providers in search/extract dispatch and tests pageprune-style routing; although recorded as the best fix for [Bug]: Plugin-registered web providers can't be selected via web.extract_backend #57581, it was closed in favor of the canonical path implemented by fix(web_tools): honor plugin-registered provider availability at the tool gate #57779.
• fix(web_tools): honor plugin-registered provider availability at the tool gate #57779 [merged] best fix — (+249/-9) — n/a: Merged reference implementation centralizing registry-aware availability, shared/per-capability selection, and tool-gate coverage, and the recorded best fix for the custom-provider allowlist issue class.
• fix(tools): add actionable guidance to web_search/web_extract error messages #58359 best fix — (+10/-5) — n/a: Adds actionable setup guidance, but its keep_open contributor review finds the cache/web and unconditional browser_navigate advice inapplicable, requests focused tests, and leaves an unrelated Himalaya edit to remove.
• fix(web): load plugins before resolving extract backend #67110 duplicate — (+8/-1) — n/a: Moves discovery before extract-backend resolution, fixing that dispatcher path; its keep_open contributor review requires a cold-registry custom-provider test, while fix(web-tools): ensure plugins loaded before backend selection for plugin-registered providers #67309 covers the shared sibling gates with such visible tests.
• fix(web-tools): ensure plugins loaded before backend selection for plugin-registered providers #67309 partial — (+285/-0) — n/a: Adds discovery at _get_backend() and _is_backend_available(), with cold-registry ordering, selection, negative, reentrancy, and real-discovery tests that address the visible keep_open review's original missing-test objection.
• fix(web): ensure plugin discovery runs before web provider resolution #69144 [closed] partial — (+25/-0) — n/a: Adds discovery in registry resolution and dispatchers; it was closed because the maintainer-bot cites fix(web): ensure plugin discovery before web_*_tool registry lookups #34563 and 6e179c4 as the existing implementation.
• fix(web-tools): lazy plugin discovery in registry access functions #71791 partial — (+36/-1) — n/a: Lazily discovers plugins from registry helpers, but its keep_open contributor review requires reuse of _ensure_web_plugins_loaded(), cold-registry tests, and check_web_api_key() coverage; its ordering scope overlaps fix(web-tools): ensure plugins loaded before backend selection for plugin-registered providers #67309.
• fix(web): load user extract plugins before backend resolution #72646 partial — (+91/-1) — n/a: Discovers non-legacy extract plugins before capability selection and separately fixes nested error:null display classification; its keep_open contributor review confirms check_web_api_key() remains a cold-registry sibling path requiring the same guard and a regression test.

Duplicates

#27584, #27700, #28202, and #69144 overlap on discovery-before-dispatch, with #34563 as the merged reference; #31829, #31887, #32465, #33516, #33902, #36094, #38375, #52057, #56201, and #56761 overlap with the provider-gate class implemented by #57779. #67110, #67309, #71791, and the web-selection portion of #72646 overlap on discovery-before-selection.

Suggested consolidation

Keep #67309 open with a salvage path for the shared-gate ordering fix and its corrected cold-registry tests, subject to an independent run against current main. Author action on #72646: add the contributor-requested discovery guard and cold-registry test for check_web_api_key(), then split out the distinct nested error:null display fix; despite the keep_open reviews on #67110 and #71791, their diffs are narrower or incomplete relative to #67309, so they can be closed as duplicates of #67309 after preserving any useful test cases, while #58359 should remain open only after replacing the unsupported cache/browser guidance, adding focused tests, and removing the unrelated Himalaya edit.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I27683(["issue #27683 (closed)"])
    subgraph Dup67110 ["PRs duplicating each other"]
        P67110["PR #67110 (open)"]
        P72646["PR #72646 (open)"]
    end
    P72646 -.->|partial| I27683
    class I27683 closed
    class P67110 open
    class P72646 open
    class P72646 target
    click I27683 "https://github.com/NousResearch/hermes-agent/issues/27683"
    click P67110 "https://github.com/NousResearch/hermes-agent/pull/67110"
    click P72646 "https://github.com/NousResearch/hermes-agent/pull/72646"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 24 pull requests and 8 issues in this complex. Each diff was read against this issue; Assessment working set: 147 kB of PR diffs, 118 kB of issue/PR text, 64 kB of discussion (79 comments), 96 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.