feat(web): add Keenable as a native web search & extract provider (keyless-capable) by IlyaGusev · Pull Request #49758 · NousResearch/hermes-agent

IlyaGusev · 2026-06-20T18:38:07Z

What does this PR do?

Adds Keenable (low-latency web search + page-to-markdown fetch for agents) as a native Web Search & Extract provider, following the plugin architecture from #25182 and the provider precedents in #29042 (xAI) / #46559 (iFlow).

Users select Keenable from the Web Search & Extract provider picker (or web.backend: keenable in config.yaml). It maps:

WebSearchProvider.search() → GET /v1/search?query= (result count applied client-side)
WebSearchProvider.extract() → GET /v1/fetch?url=

against https://api.keenable.ai (endpoints/DTOs per the published OpenAPI spec at docs.keenable.ai/api-reference). The X-API-Key header is sent when a key is configured; requests also send X-Keenable-Title: Hermes for client attribution.

The PR also ships two optional, separable surfaces for the same vendor (see Scope below): an optional CLI skill and an MCP catalog entry.

Motivation

Keenable fits the Web Search & Extract provider layer (search + page fetch in one provider, no browser sessions). Keyless, but opt-in. Keenable's free tier works without a key: when KEENABLE_API_KEY is unset the provider calls the /public endpoint variants (rate-limited) and omits X-API-Key, mirroring Keenable's official MCP client. To avoid the silent-default issue the keyless-Parallel revert (#46350) addressed, keenable is never auto-selected — is_available() stays key-gated so it is excluded from the no-credential fallback, and it serves keyless only when explicitly chosen via web.backend / web.*_backend. A KEENABLE_API_KEY raises rate limits.

Scope note (happy to split)

This PR bundles three logical pieces for one vendor. If reviewers prefer one-change-per-PR, I'll split into:

Native provider — plugins/web/keenable/ + core wiring (the primary change).
Optional CLI skill — optional-skills/research/keenable-cli/.
MCP catalog entry — optional-mcps/keenable/.

Type of Change

✨ New feature (non-breaking change that adds functionality)
🎯 New skill (optional, not bundled)

Changes Made

Native provider

plugins/web/keenable/{plugin.yaml,__init__.py,provider.py} — KeenableWebSearchProvider (search + extract), keyless /public fallback, KEENABLE_API_URL override.
tools/web_tools.py — register keenable in the configured-backend sets, auto-detect candidate, _is_backend_available, and tool-metadata env vars.
hermes_cli/config.py — KEENABLE_API_KEY in OPTIONAL_ENV_VARS + the API-key display/allowlist lists.
agent/web_search_registry.py — added to the legacy fallback preference.

Optional skill + MCP catalog

optional-skills/research/keenable-cli/SKILL.md — CLI skill (HARDLINE-compliant: 56-char description, modern section order).
optional-mcps/keenable/manifest.yaml — remote HTTP MCP catalog entry (api.keenable.ai/mcp).

Packaging / attribution / docs

pyproject.toml — data-files target so the MCP manifest ships in the wheel.
scripts/release.py — AUTHOR_MAP entry.
website/docs/** + website/i18n/zh-Hans/** — Keenable added to provider/env-var tables (EN + zh).

Tests

tests/tools/conftest.py + tests/plugins/web/test_web_search_provider_plugins.py — existing provider-set/capability assertions updated to include keenable.

How to Test

# Provider discovery + capability + ABC conformance (no network)
uv run --extra dev pytest tests/plugins/web/test_web_search_provider_plugins.py -q
# MCP catalog + packaging
uv run --extra dev pytest tests/hermes_cli/test_mcp_catalog.py tests/test_packaging_metadata.py -q
# Live smoke (keyless works; export KEENABLE_API_KEY to raise limits)
hermes config set web.backend keenable
hermes -q "search the web for the latest on AI agents and summarize the top result"

Checklist

I've read the Contributing Guide
Commits follow Conventional Commits
PR contains only Keenable-related changes
pytest tests/ -q locally — not run by the author locally (no dev env); CI is the gate. Provider request/response shaping verified via injected-httpx checks (keyed + keyless paths).
Dedicated provider test (tests/tools/test_web_providers_keenable.py) — not yet added; existing provider-set tests updated instead. Happy to add to match the xAI/iFlow pattern.
Tested on Linux / Python 3.11 (request shapes + normalization)
Cross-platform: pure-Python httpx; skill declares all three platforms; MCP is a remote URL
Docs updated (EN + zh-Hans provider/env-var tables)
No new PyPI dependencies (reuses existing httpx)

Notes for maintainers

Endpoints/DTOs follow Keenable's published OpenAPI spec (docs.keenable.ai/api-reference). Live keyless smoke test passed against api.keenable.ai (/v1/search/public + /v1/fetch/public): search returns ranked results, fetch returns page markdown. The smoke test also caught and fixed a spec/live mismatch — /v1/search takes no count param, so limit is applied client-side. The keyed X-API-Key path is covered by injected-httpx checks (no raw API key on hand for a live keyed run). Squash-merge recommended.

teknium1

Thanks for building this against the web-provider interface. The native-provider portion needs a scope change before it can be considered.

Problems

plugins/web/keenable/ is a new in-tree third-party vendor integration, which conflicts with the standalone-plugin policy in AGENTS.md:126-136. The optional skill and MCP manifest are separable contributions.
plugins/web/keenable/provider.py:64 unconditionally sends X-Keenable-Title: Hermes; AGENTS.md:118-121 requires an explicit generic opt-in for outbound attribution tagging.
plugins/web/keenable/provider.py:54-59 reads credentials directly from os.getenv, unlike the config-aware provider helper at agent/web_search_provider.py:59-81.
The PR's provider test changes are inventory assertions only (tests/plugins/web/test_web_search_provider_plugins.py:81-132); request shaping and keyless/keyed behavior remain untested.

Suggested changes

Preserve the optional skill/MCP work if desired, but distribute the native provider as a standalone plugin.
Remove or opt-in-gate attribution, use get_provider_env, and add mocked provider behavior tests.

Current tools/web_tools.py:231-327 already resolves registered providers generically, consistent with the #31873 discussion, so the vendor-specific backend-gate edits need a fresh port rather than a direct salvage.

This is an automated hermes-sweeper review.

teknium1 · 2026-07-14T22:10:55Z

+
+
+def _keenable_headers(api_key: str) -> Dict[str, str]:
+    """Request headers; ``X-API-Key`` only when a key is present (keyless otherwise)."""


Use agent.web_search_provider.get_provider_env("KEENABLE_API_KEY") here and in is_available(). Direct os.getenv() bypasses Hermes-managed .env lookup for gateway, delegate-child, and subprocess contexts; see agent/web_search_provider.py:59-81.

teknium1 · 2026-07-14T22:10:55Z

+def _endpoint(base_url: str, path: str, api_key: str) -> str:
+    """Keyless calls hit the ``/public`` variant (no auth, rate-limited)."""
+    return f"{base_url}{path}" if api_key else f"{base_url}{path}/public"
+


This unconditionally tags every outbound request with Hermes attribution. AGENTS.md:118-121 requires a generic user-facing opt-in before adding outbound attribution or third-party identifier tagging; remove this header or implement that prerequisite.

teknium1 · 2026-07-14T22:10:55Z

@@ -96,6 +98,7 @@ def test_all_seven_plugins_present_in_registry(self) -> None:
            ("parallel", True, True),


Please add a dedicated mocked provider test file covering keyed and keyless request URLs/headers, search normalization and truncation, extract result/error shapes, and config-managed credential lookup. This table only verifies inventory and capability declarations.

Native web-search provider moved out to a standalone pip plugin (hermes-keenable-web) per the standalone-plugin policy. This keeps only the two separable, in-tree parts: the optional MCP catalog entry and the research skill.

ilya-bogin-keenable · 2026-07-18T14:07:09Z

Reworked this per the review.

I pulled the in-tree provider and all its wiring. It now ships as a standalone pip plugin, hermes-keenable-web (on PyPI), which is the standalone-plugin path AGENTS.md asks for. That plugin reads creds through get_provider_env, has mocked search/extract behavior tests, and registers via the hermes_agent.plugins entry point, so it lands in the Search Provider picker once installed. The attribution header question goes away with the in-tree provider too; the standalone build keeps to the same policy.

What's left in this PR is just the two separable pieces you flagged: the MCP catalog entry (optional-mcps/keenable/) and the research skill (optional-skills/research/keenable-cli/). Neither touches the provider path. Happy to adjust either if something's off.

ilya-bogin-keenable · 2026-08-04T05:22:01Z

@teknium1 could you take a look please?

IlyaGusev requested a review from a team June 20, 2026 18:38

alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/web Web search and extraction comp/plugins Plugin system and bundled plugins labels Jun 20, 2026

IlyaGusev changed the title ~~feat(web): add Keenable as a native web search & extract provider~~ feat(web): add Keenable as a native web search & extract provider (keyless-capable) Jun 20, 2026

Pauliehedron mentioned this pull request Jul 3, 2026

[Bug]: check_web_api_key() hardcodes built-in backends — third-party web search plugins silently disabled #31873

Closed

1 task

teknium1 reviewed Jul 14, 2026

View reviewed changes

ilya-bogin-keenable force-pushed the feat/keenable-integration branch from 1084011 to 1f7668d Compare July 18, 2026 14:04

Navigation Menu

Uh oh!

feat(web): add Keenable as a native web search & extract provider (keyless-capable) - #49758

feat(web): add Keenable as a native web search & extract provider (keyless-capable)#49758
IlyaGusev wants to merge 1 commit into
NousResearch:mainfrom
keenableai:feat/keenable-integration

IlyaGusev commented Jun 20, 2026 •

edited

Loading

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

ilya-bogin-keenable commented Jul 18, 2026

Uh oh!

ilya-bogin-keenable commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants



		def _keenable_headers(api_key: str) -> Dict[str, str]:
		"""Request headers; ``X-API-Key`` only when a key is present (keyless otherwise)."""

		@@ -96,6 +98,7 @@ def test_all_seven_plugins_present_in_registry(self) -> None:
		("parallel", True, True),

Uh oh!

Conversation

IlyaGusev commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Scope note (happy to split)

Type of Change

Changes Made

How to Test

Checklist

Notes for maintainers

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

ilya-bogin-keenable commented Jul 18, 2026

Uh oh!

ilya-bogin-keenable commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

IlyaGusev commented Jun 20, 2026 •

edited

Loading