feat(firecrawl): add Keyless (no-API-key) mode via lightweight HTTP transport

[hashbender]

Summary

Firecrawl launched Keyless mode in June 2026,
allowing search and scrape calls without an API key — no account needed,
1,000 credits/month free. The cloud API accepts unauthenticated requests at the
same endpoints.

The Firecrawl Python SDK unconditionally requires api_key at construction,
making it unsuitable for this mode. Rather than forking the SDK or adding a
stub key, this PR introduces a lightweight HTTP transport (_KeylessFirecrawlClient)
that mirrors the SDK's method signatures so the existing extract loop works
unchanged.

This builds on two recent Firecrawl provider improvements that this Keyless
client inherits naturally:

• feat(web_extract): truncate-and-store instead of LLM summarization NousResearch/hermes-agent#54843 (truncate-and-store) — eliminated the LLM summarizer bottleneck;
  Firecrawl's clean markdown now goes straight to cache.
• fix(web): re-check Firecrawl final URLs for SSRF (salvage #35840) NousResearch/hermes-agent#56143 (SSRF re-check) — added redirect-target safety validation in the
  same extract path our Keyless client shares (salvage fix(web): re-check Firecrawl final URLs for SSRF NousResearch/hermes-agent#35840 by @zapabob).

With these in place, the missing piece for a true zero-setup experience is a
transport that works without credentials — which this PR adds.

Changes

plugins/web/firecrawl/provider.py — +56 lines

1. _KeylessFirecrawlClient (lines 77–121)

httpx-based transport that calls POST /v1/search and POST /v1/scrape.
Implements search(query, limit) -> dict / scrape(url, formats) -> dict
with the same surface as the SDK client, so the existing extract loop
(including the SSRF re-check from NousResearch#56143) works unchanged.

2. Dispatch in _get_firecrawl_client() (lines 305–313)

When resolved kwargs contain no api_key (i.e. FIRECRAWL_API_URL is set
without FIRECRAWL_API_KEY), returns a _KeylessFirecrawlClient instead of
constructing an SDK client.

No other changes required

_get_direct_firecrawl_config, check_firecrawl_api_key, and is_available
all already accept the FIRECRAWL_API_URL-only config as valid — no
modifications needed.

Usage

# .env — no FIRECRAWL_API_KEY needed
FIRECRAWL_API_URL=https://api.firecrawl.dev

# configure backend
hermes config set web.backend firecrawl
hermes config set web.search_backend firecrawl
hermes config set web.extract_backend firecrawl

Test Plan

• Manual verification with live Firecrawl Keyless API

Client: <_KeylessFirecrawlClient api_url=https://api.firecrawl.dev>
Search: 3 results — success
Scrape: 5660 chars, title extracted — success

• Existing SDK path unchanged (regression test) — the keyless branch is only
  taken when api_key is absent from kwargs; all existing configs with
  FIRECRAWL_API_KEY continue using the SDK path.

Notes for Reviewers

• Why not modify the SDK? The Firecrawl SDK requires api_key at the
  constructor level. Adding keyless support there would require an upstream
  change in firecrawl-py. This keeps the integration self-contained: ~45
  lines of transport code that can be cleanly removed if the SDK later
  supports keyless natively.
• No new dependencies — uses httpx, which Hermes already depends on.
• Signed off the same SSRF checks from fix(web): re-check Firecrawl final URLs for SSRF (salvage #35840) NousResearch/hermes-agent#56143 apply — _KeylessFirecrawlClient
  feeds into the same _extract_scrape_payload / is_safe_url(final_url)
  pipeline.

---

Mirror-of: NousResearch#57151
NousResearch#57151