feat(xai): xai_web_search tool — live web search via xAI Responses API web_search

[Julientalbot]

Summary

Adds xai_web_search — a tool that runs a live web search via xAI's Responses API web_search built-in tool, returning an answer grounded in citations.

This is the web counterpart of x_search (#14541): x_search targets X / Twitter, xai_web_search targets the open web. Together they cover both surfaces of xAI's two-tool search ecosystem (per the xAI API reference: "only functions and web search are supported as tools").

Why

Hermes already has a generic web_search tool (in tools/web_tools.py) backed by various third-party providers (Exa, Firecrawl, Parallel-Web). That's the right default for portability.

But when an xAI API key is available, the xAI-native web_search is the more direct path:

• Single API call (no third-party intermediary).
• Citations come back in the same response, anchored with start_index / end_index offsets into the answer text.
• Pricing and rate limits are governed by xAI directly — predictable for users already paying for xAI.

This tool gives Hermes profiles that have XAI_API_KEY set the option of routing search through xAI without changing the existing web_search default.

Changes (5 files, +743/-0)

tools/xai_web_search_tool.py (new, ~395 LoC)

Implementation symmetric with the rebased tools/x_search_tool.py from #14541 — the two share the same Responses-API-as-search-backend pattern.

web_search_tool(
    query: str,                              # required
    allowed_websites: list[str] | None = None,  # max 10, hostnames; mutually exclusive
    excluded_websites: list[str] | None = None, # max 10, hostnames
    from_date: str = "",                     # ISO YYYY-MM-DD
    to_date: str = "",                       # ISO YYYY-MM-DD
    country: str = "",                       # ISO alpha-2; uppercased
) -> str  # JSON-encoded {success, provider, tool, model, query, answer, citations, inline_citations}

Key design choices:

• Hostname normalization: https://www.nytimes.com/section/foo → nytimes.com (strips protocol, www., path) so the user can paste any URL form.
• Citations on two channels: top-level data.citations (xAI's flat list of sources used) + inline url_citation annotations on output[*].content[*].annotations with start_index / end_index offsets so callers can hyperlink the answer text.
• Retry policy matches x_search: retry on 5xx and read-timeout / connection errors with exponential backoff capped at 5s; no retry on 4xx auth errors.
• Configurable via config.yaml:

  web_search:
    model: grok-4.20-reasoning
    timeout_seconds: 240
    retries: 3

• Reuses tools.xai_http.hermes_xai_user_agent for User-Agent.
• Self-registers via tools.registry.registry.register, gated on XAI_API_KEY.
• Registered as xai_web_search because the bare web_search name is already owned by tools/web_tools.py (the generic, provider-neutral search). Profiles can enable both — the agent picks per-call.

tests/tools/test_xai_web_search_tool.py (new, ~290 LoC)

33 unit tests with a requests.post monkeypatch fake (no real network):

• Requirements / schema: check_web_search_requirements (with / without / blank API key); required query; optional params advertised.
• _normalize_websites: strips https:// / http:// / www. / paths; drops empty entries; rejects lists > 10; handles None.
• Argument validation: empty query, missing API key, allowed_websites + excluded_websites mutually exclusive — all surface as success: false JSON.
• Body construction: default model, /responses endpoint (not /chat/completions), allowed-/excluded-websites threading, date range, country uppercased, minimal tool_def when no options, complete headers (Bearer, JSON, User-Agent).
• Response parsing: answer extraction; top-level citations passthrough; inline url_citation annotation extraction; legacy output_text fallback; multi-piece text concatenation; non-url_citation annotation types skipped.
• HTTP errors: 401 surfaces immediately; 500 retries then succeeds when retries available; 500 exhausts retries; 4xx never retries (saves the user from compounding auth errors).

Wiring

• toolsets.py: xai_web_search added to _HERMES_CORE_TOOLS; new xai_web_search toolset.
• hermes_cli/tools_config.py: new xai_web_search entry in CONFIGURABLE_TOOLSETS.
• tests/tools/test_registry.py: tools.xai_web_search_tool added to the manual snapshot list (same convention as feat(xai): add x_search tool — search X via xAI Responses API #14541, feat(xai): add video generation tool — generate/edit/extend via xAI grok-imagine-video #14543, feat(xai): xai_deferred_chat tool — long-running completions via submit + poll #23329, feat(xai): xai_batch_chat tool — submit many completions to xAI Batch API #23333, feat(xai): xai_responses_chat tool — stateful Responses API surface (store, previous_response_id, max_turns) #23343).

Validation

• pytest tests/tools/test_xai_web_search_tool.py tests/tools/test_registry.py → 64/64 passing locally on top of current main.

Backward compatibility

• Pure addition. The existing generic web_search tool in tools/web_tools.py is untouched.
• Tool is check_fn-gated on XAI_API_KEY — invisible to users without an xAI key.
• xai_web_search is not in any default toolset roster; users opt in via tools_config or by enabling the toolset in their profile.
• A profile can enable both web_search and xai_web_search simultaneously; the agent picks per-call based on which is more appropriate for the query.

Scope deliberately not in this PR

• Default-routing logic that picks xai_web_search over web_search when xAI is available. That's a per-profile policy decision, not a tool-level concern.
• Streaming (stream=true on the underlying Responses call). Web search is typically a single-shot answer; streaming would mostly mask the citation block until the end.
• max_search_results as a tool-level argument. Currently controlled via the search_parameters.max_search_results body field if needed; happy to expose if the maintainer wants it.

[Julientalbot]

Closing this older xAI web search PR in favor of #27023, which supersedes it with a fresh main base, the current xAI Responses API web_search shape (filters.allowed_domains / filters.excluded_domains, max 5), refreshed tests, and active CI.