feat(firecrawl): add Keyless (no-API-key) mode via lightweight HTTP transport

[luxles]

Summary

Firecrawl launched Keyless mode in June 2026,
allowing search and scrape calls without an API key — no account needed,
1,000 credits/month free. The cloud API accepts unauthenticated requests at the
same endpoints.

The Firecrawl Python SDK unconditionally requires api_key at construction,
making it unsuitable for this mode. Rather than forking the SDK or adding a
stub key, this PR introduces a lightweight HTTP transport (_KeylessFirecrawlClient)
that mirrors the SDK's method signatures so the existing extract loop works
unchanged.

This builds on two recent Firecrawl provider improvements that this Keyless
client inherits naturally:

• feat(web_extract): truncate-and-store instead of LLM summarization #54843 (truncate-and-store) — eliminated the LLM summarizer bottleneck;
  Firecrawl's clean markdown now goes straight to cache.
• fix(web): re-check Firecrawl final URLs for SSRF (salvage #35840) #56143 (SSRF re-check) — added redirect-target safety validation in the
  same extract path our Keyless client shares (salvage fix(web): re-check Firecrawl final URLs for SSRF #35840 by @zapabob).

With these in place, the missing piece for a true zero-setup experience is a
transport that works without credentials — which this PR adds.

Changes

plugins/web/firecrawl/provider.py — +56 lines

1. _KeylessFirecrawlClient (lines 77–121)

httpx-based transport that calls POST /v1/search and POST /v1/scrape.
Implements search(query, limit) -> dict / scrape(url, formats) -> dict
with the same surface as the SDK client, so the existing extract loop
(including the SSRF re-check from #56143) works unchanged.

2. Dispatch in _get_firecrawl_client() (lines 305–313)

When resolved kwargs contain no api_key (i.e. FIRECRAWL_API_URL is set
without FIRECRAWL_API_KEY), returns a _KeylessFirecrawlClient instead of
constructing an SDK client.

No other changes required

_get_direct_firecrawl_config, check_firecrawl_api_key, and is_available
all already accept the FIRECRAWL_API_URL-only config as valid — no
modifications needed.

Usage

# .env — no FIRECRAWL_API_KEY needed
FIRECRAWL_API_URL=https://api.firecrawl.dev

# configure backend
hermes config set web.backend firecrawl
hermes config set web.search_backend firecrawl
hermes config set web.extract_backend firecrawl

Test Plan

• Manual verification with live Firecrawl Keyless API

Client: <_KeylessFirecrawlClient api_url=https://api.firecrawl.dev>
Search: 3 results — success
Scrape: 5660 chars, title extracted — success

• Existing SDK path unchanged (regression test) — the keyless branch is only
  taken when api_key is absent from kwargs; all existing configs with
  FIRECRAWL_API_KEY continue using the SDK path.

Notes for Reviewers

• Why not modify the SDK? The Firecrawl SDK requires api_key at the
  constructor level. Adding keyless support there would require an upstream
  change in firecrawl-py. This keeps the integration self-contained: ~45
  lines of transport code that can be cleanly removed if the SDK later
  supports keyless natively.
• No new dependencies — uses httpx, which Hermes already depends on.
• Signed off the same SSRF checks from fix(web): re-check Firecrawl final URLs for SSRF (salvage #35840) #56143 apply — _KeylessFirecrawlClient
  feeds into the same _extract_scrape_payload / is_safe_url(final_url)
  pipeline.

[alt-glitch]

This was generated by AI during triage.

Competing with #50659 for issue #49912 (Firecrawl keyless mode). Both add a _KeylessFirecrawlClient to provider.py, but the dispatch trigger differs: #50659 routes to keyless only on explicit Firecrawl selection, while this PR routes whenever no api_key is present. Related, not duplicates (same code site, different mechanism); closed #52122 was a third, SDK-native approach. Flagging the pair for a maintainer to pick (earliest = #50659).

[luxles]

Relationship to #50659

This PR and #50659 (by @LeonSGP43) both add a _KeylessFirecrawlClient to provider.py, but the trigger conditions are complementary, not duplicative:

Scenario	#50659 covers?	#57151 covers?
User runs hermes config set web.backend firecrawl, no env vars	✅ Yes — config-selection triggers keyless	❌ No — requires FIRECRAWL_API_URL
User sets FIRECRAWL_API_URL but has no API key (self-hosted / custom endpoint)	❌ No — sends bare FIRECRAWL_API_URL to SDK, which crashes	✅ Yes — api_key absent in kwargs triggers keyless

The ideal resolution combines both triggers in _get_direct_firecrawl_config():

1. FIRECRAWL_API_KEY + optional URL → SDK mode (existing)
2. FIRECRAWL_API_URL set without FIRECRAWL_API_KEY → keyless (feat(firecrawl): add Keyless (no-API-key) mode via lightweight HTTP transport #57151's path)
3. No env vars at all but Firecrawl selected in config → keyless (fix(web): allow explicit Firecrawl keyless mode #50659's path)

Minor technical difference: #50659 uses /v2/ endpoints while this PR uses /v1/. Either works — the cloud accepts both.

Happy to close this PR in favor of a combined approach if the maintainer prefers. Leaving both open for maintainer pickup.

[teknium1]

Thanks for isolating a raw HTTP transport from the keyed SDK path.

Problems

• The added branch only runs after _get_direct_firecrawl_config() produces URL-only kwargs. On current main, no-key/no-URL returns None at plugins/web/firecrawl/provider.py:130, and check_firecrawl_api_key() stays unavailable at line 176. This PR therefore does not reach the no-setup cloud-keyless flow described in issue #49912; it covers only the URL-only case. The discussion correctly identifies #50659 as the complementary explicit-selection path.
• The diff has no tests for the new transport or dispatch. Existing URL-only availability coverage is at tests/tools/test_web_tools_config.py:600-603, but it does not verify construction or HTTP behavior.
• plugins/web/firecrawl/plugin.yaml:3 and provider.py:605-615 still describe URL-only use as self-hosted/paid rather than keyless cloud.

Suggested changes

• Combine the URL-only branch with an explicit Firecrawl-selection keyless path, or explicitly narrow the PR's scope.
• Add transport/selection tests and align setup metadata.

Automated hermes-sweeper review.

[teknium1]

This condition is only reachable when _get_direct_firecrawl_config() already returned URL-only kwargs. With no key and no URL, current main returns None at provider.py:130 and raises through the gateway path, so this does not implement no-setup keyless cloud use. Please combine this with the explicit Firecrawl-selection trigger discussed for #50659, or narrow the stated scope and add coverage for the URL-only behavior.