feat: keyless web tier is now a 5-vendor free rotation (Exa/Parallel/Tavily/Firecrawl/Keenable) with ring failover + honest doctor readiness - #90572
Merged
Conversation
- Updated the Tavily API key description to clarify that it is optional and keyless access is supported. - Modified the Tavily plugin and provider to handle requests with or without an API key, using Bearer authentication when the key is provided. - Enhanced documentation to reflect the new keyless functionality and updated environment variable descriptions. - Added tests to ensure correct behavior for both keyed and keyless requests.
- Added support for keyless Tavily integration in the onboarding flow, allowing it to be recognized as available without an API key.
…ic cloud API Salvaged from #50659 by @LeonSGP43 onto current main (the client resolver was rewritten for strict-selection semantics since the PR; reapplied the keyless mode as a third client_mode inside the new resolver). An explicit firecrawl selection with no FIRECRAWL_API_KEY / FIRECRAWL_API_URL now routes through a minimal REST client (v2 search + scrape, no Authorization header) instead of erroring. Unconfigured installs never route here — the keyless path requires the explicit selection. Fixes #49912.
- Cross-vendor failover: when Exa's or Parallel's keyless free tier returns a rate-limit-shaped error, the request retries once on the other vendor's free endpoint (search + whole-batch extract). Result notes served_by; a peer pinned to its paid tier is never used; non-throttle errors never fail over. - Docs: failover note + Tavily/Firecrawl keyless-when-selected rows. - Firecrawl keyless test expectations aligned with the keyless tier.
#78412) Salvaged from #78434 by @Slobaka (also the issue reporter; earlier than the competing #78436). hermes doctor no longer paints a green web check when the explicitly selected provider cannot initialize — web splits into per-capability rows (web search / web extract) resolved through the same registry resolvers the dispatchers use, with readiness from a true availability probe (_provider_is_ready). Keyless-tier integration on top of the salvage: - _provider_is_ready counts is_keyless_available() as ready — keyless mode is a working state, not a misconfiguration (zero-config installs and selected-keyless Tavily/Firecrawl show ok, not warn) - Tavily/Firecrawl gain is_keyless_available() (True only when explicitly selected — they stay out of the zero-config fallback) - doctor triggers plugin discovery before reading the registry (fresh doctor processes saw an empty registry and warned on everything) E2E: searxng-selected-without-URL warns (the #78412 repro); zero-config, tavily-keyless, firecrawl-keyless all read ok; parallel pinned paid without a key warns.
…irecrawl-failover
Contributor
૮ >ﻌ< ა ci reviewran on e7e3885 — test: pin ring entry vendor in provider-routing tests (ring
|
alt-glitch
added
type/feature
Collaborator
…route The salvaged #50659 behavior makes 'firecrawl selected, no creds' a WORKING keyless state, so the old expectation (hard error naming FIRECRAWL_API_KEY) is stale. The test now mocks httpx and asserts the request routes to api.firecrawl.dev with results returned — still proving keyless Tavily can't silently take over, which was the test's point. Also stops the test making a real network call in CI.
…-parse)" This reverts commit 87944ad.
…ly, Firecrawl, Keenable) Fresh installs with zero web credentials now rotate web_search/ web_extract across FIVE vendors' public free tiers — Exa, Parallel, Tavily, Firecrawl, Keenable — instead of a 2-vendor 50/50 split, with next-in-line ring failover on rate limits (multi-hop until a vendor serves or the ring is exhausted; served_by marks the actual vendor). - plugins/web/keenable/: new bundled provider (search via /v1/search, fetch via /v1/fetch; keyed Bearer or keyless with the mandatory X-Keenable-Title app header). Credit: integration proposed by Ilya Gusev (Keenable) in #49758; Free/Paid picker rows included. - keyless_mcp: tavily/firecrawl/keenable keyless search+extract wrappers, _KEYLESS_RING + per-process round-robin cursor (seeded by the random session id, advances per unpinned request), pinned-vendor entry (pin = start there; rotation off), paid-pinned vendors excluded from the ring entirely. - Tavily/Firecrawl providers route keyless traffic through the ring; both are now default-on ring members (no longer selection-gated). - web_tools/registry: keenable in backend sets, auto-detect, availability probes; _keyless_preference() delegates to the ring cursor. - KEENABLE_API_KEY in OPTIONAL_ENV_VARS; docs updated (ring semantics). Live E2E: all 10 vendorXcapability paths (5 search + 5 extract) served real results keyless; rotation cycled all five vendors over 5 dispatch calls; double-throttle failover walked exa->parallel->tavily.
…irecrawl-failover
…made direct-callable mocks stale)
teknium1
changed the title
This was referenced Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every keyless-capable web vendor now works keyless in Hermes, throttled keyless requests automatically fail over to the peer free tier, and
hermes doctorreports the web tools' real readiness instead of a false green (#78412).Salvages two contributor PRs onto current main with authorship preserved:
X-Tavily-Access-Mode: keyless+X-Client-Name: hermes-agent)Changes
plugins/web/keyless_mcp.py: cross-vendor failover — a rate-limit-shaped error on Exa's/Parallel's keyless tier retries once on the peer's free endpoint (search + whole-batch extract;served_bymarks the actual vendor; a peer pinnedpaidis never used; non-throttle errors never fail over)plugins/web/tavily/*: keyless headers/keyed Bearer split, error bodies surfaced on HTTP ≥400, optional-key setup schema (salvage, adapted to current strict-selection main)plugins/web/firecrawl/*:_KeylessFirecrawlClient(v2 search/scrape, no auth header) as a thirdclient_modeinside the strict-selection resolver; keyless requires explicit selectiontools/web_tools.py:_provider_is_ready()(doctor/readiness gate) counts keyed OR keyless-capable as ready;check_web_api_key()routes through itplugins/web/{tavily,firecrawl}/provider.py:is_keyless_available()— True only when explicitly selected, keeping both OUT of the automatic zero-config fallback (that stays Exa/Parallel 50/50)hermes_cli/doctor.py: per-capability web rows + plugin-discovery kick (fresh doctor processes saw an empty registry and warned on everything)hermes_cli/nous_subscription.py: selected-keyless Tavily counts as web-activeValidation
web.backend: tavily, no keyweb.backend: firecrawl, no keyserved_bysetInfographic